Forge Home

hadoop_lib

Common puppet library for Hadoop modules

18,229 downloads

996 latest version

4.6 quality score

We run a couple of automated
scans to help you access a
module's quality. Each module is
given a score based on how well
the author has formatted their
code and documentation and
modules are also checked for
malware using VirusTotal.

Please note, the information below
is for guidance only and neither of
these methods should be considered
an endorsement by Puppet.

Version information

  • 0.5.0 (latest)
  • 0.4.0
  • 0.3.0
  • 0.2.1
  • 0.2.0
  • 0.1.0
released Dec 21st 2020
This version is compatible with:
  • , Fedora, , , ,

Start using this module

  • r10k or Code Manager
  • Bolt
  • Manual installation
  • Direct download

Add this module to your Puppetfile:

mod 'cesnet-hadoop_lib', '0.5.0'
Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add cesnet-hadoop_lib
Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install cesnet-hadoop_lib --version 0.5.0

Direct download is not typically how you would use a Puppet module to manage your infrastructure, but you may want to download the module in order to inspect the code.

Download

Documentation

cesnet/hadoop_lib — version 0.5.0 Dec 21st 2020

Hadoop Puppet Helper Library

Build Status Puppet Forge

Table of Contents

  1. Module Description - What the module does and why it is useful
  2. Setup - The basics of getting started with hadoop_lib
  3. Usage - Beginning, configuration options, and additional functionality
  4. Reference - An under-the-hood peek at what the module is doing and how
  5. Limitations - OS compatibility, etc.
  6. Development - Guide for contributing to the module

Module Description

Common puppet library for Hadoop and its addons. It manages alternatives as used in BigTop-based Hadoop distributions and provides necessary environment variables.

Setup

What hadoop_lib affects

  • Files created (env class):
  • /etc/profile.d/hadoop.sh
  • /etc/profile.d/hadoop.csh
  • Links created (jdbc resource type)
  • Alternatives (postinstall resource type):
  • /etc/<NAME>/conf: it can be switched to the new alternative, so original configuration is kept intact

Usage

Example: alternatives

Package['hive']
->
::hadoop_lib::postinstall { 'hive':
  alternatives => 'cluster',
}

Example: environment

::hadoop_lib::env

Example: MySQL JDBC jar

class { 'mysql::bindings':
  java_enable => true,
}
->
::hadoop_lib::jdbc { '/usr/lib/hive/lib':
  db => 'mysql',
}

Example: array search function

$_myid = array_search($hostnames, $::fqdn)

Reference

###Classes

hadoop_lib::env

Parameters

(none)

###Resources

hadoop_lib::postinstall

Parameters

#####alternatives

Switches the alternatives used for the configuration. Default: 'cluster' (Debian) or undef.

It can be used only when supported (for example with Cloudera distribution).

hadoop_lib::jdbc

Link JDBC jar file from standard location. Jar files must be installed.

Example of using puppetlabs modules for jar files installation:

  • puppetlabs-mysql: mysql::bindings class with java_enable parameter set
  • puppetlabs-postgresql: postgresql::lib::java class

Parameters

#####db

Database type. Default: undef.

Values:

  • mysql, mariadb: MySQL/MariaDB,
  • postgresql: PostgreSQL

###Functions

array_search

Searches a value in the array and returns the index starting from 1.

Parameters

#####array

#####value

##Development