hadoop_lib

Common puppet library for Hadoop modules

18,557 downloads

1,122 latest version

4.6 quality score

Version information

  • 0.5.0 (latest)
  • 0.4.0
  • 0.3.0
  • 0.2.1
  • 0.2.0
  • 0.1.0
released Dec 21st 2020
This version is compatible with:
  • CentOS
    , Fedora,
    Debian
    ,
    RedHat
    ,
    Scientific
    ,
    Ubuntu

Start using this module

  • r10k or Code Manager
  • Bolt
  • Manual installation
  • Direct download

Add this module to your Puppetfile:

mod 'cesnet-hadoop_lib', '0.5.0'
Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add cesnet-hadoop_lib
Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install cesnet-hadoop_lib --version 0.5.0

Direct download is not typically how you would use a Puppet module to manage your infrastructure, but you may want to download the module in order to inspect the code.

Download

Documentation

cesnet/hadoop_lib — version 0.5.0 Dec 21st 2020

Hadoop Puppet Helper Library

Build Status Puppet Forge

Table of Contents

  1. Module Description - What the module does and why it is useful
  2. Setup - The basics of getting started with hadoop_lib
  3. Usage - Beginning, configuration options, and additional functionality
  4. Reference - An under-the-hood peek at what the module is doing and how
  5. Limitations - OS compatibility, etc.
  6. Development - Guide for contributing to the module

Module Description

Common puppet library for Hadoop and its addons. It manages alternatives as used in BigTop-based Hadoop distributions and provides necessary environment variables.

Setup

What hadoop_lib affects

  • Files created (env class):
  • /etc/profile.d/hadoop.sh
  • /etc/profile.d/hadoop.csh
  • Links created (jdbc resource type)
  • Alternatives (postinstall resource type):
  • /etc/<NAME>/conf: it can be switched to the new alternative, so original configuration is kept intact

Usage

Example: alternatives

Package['hive']
->
::hadoop_lib::postinstall { 'hive':
  alternatives => 'cluster',
}

Example: environment

::hadoop_lib::env

Example: MySQL JDBC jar

class { 'mysql::bindings':
  java_enable => true,
}
->
::hadoop_lib::jdbc { '/usr/lib/hive/lib':
  db => 'mysql',
}

Example: array search function

$_myid = array_search($hostnames, $::fqdn)

Reference

###Classes

hadoop_lib::env

Parameters

(none)

###Resources

hadoop_lib::postinstall

Parameters

#####alternatives

Switches the alternatives used for the configuration. Default: 'cluster' (Debian) or undef.

It can be used only when supported (for example with Cloudera distribution).

hadoop_lib::jdbc

Link JDBC jar file from standard location. Jar files must be installed.

Example of using puppetlabs modules for jar files installation:

  • puppetlabs-mysql: mysql::bindings class with java_enable parameter set
  • puppetlabs-postgresql: postgresql::lib::java class

Parameters

#####db

Database type. Default: undef.

Values:

  • mysql, mariadb: MySQL/MariaDB,
  • postgresql: PostgreSQL

###Functions

array_search

Searches a value in the array and returns the index starting from 1.

Parameters

#####array

#####value

##Development