Forge Home

Modules

Puppet

Resources

PuppetEcosystemPuppet CommunityUpdates

hadoop

Install Hadoop MapReduce Next Generation.
Project URLRSS Feed

16,083 downloads

10,505 latest version

1.5 quality score

We run a couple of automated
scans to help you access a
module's quality. Each module is
given a score based on how well
the author has formatted their
code and documentation and
modules are also checked for
malware using VirusTotal.

Please note, the information below
is for guidance only and neither of
these methods should be considered
an endorsement by Puppet.

Support the Puppet Community by contributing to this module

You are welcome to contribute to this module by suggesting new features, currency updates, or fixes. Every contribution is valuable to help ensure that the module remains compatible with the latest Puppet versions and continues to meet community needs. Complete the following steps:

  1. Review the module’s contribution guidelines and any licenses. Ensure that your planned contribution aligns with the author’s standards and any legal requirements.
  2. Fork the repository on GitHub, make changes on a branch of your fork, and submit a pull request. The pull request must clearly document your proposed change.

For questions about updating the module, contact the module’s author.

Version information

  • 0.0.18 (latest)
  • 0.0.17
  • 0.0.16
  • 0.0.15
  • 0.0.14
  • 0.0.13
  • 0.0.12
  • 0.0.11
  • 0.0.10
  • 0.0.9
  • 0.0.8
  • 0.0.7
  • 0.0.6
  • 0.0.5
  • 0.0.4
  • 0.0.3
  • 0.0.2
  • 0.0.1
released May 28th 2014

Start using this module

  • r10k or Code Manager
  • Bolt
  • Manual installation
  • Direct download

Add this module to your Puppetfile:

mod 'viirya-hadoop', '0.0.18'
Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add viirya-hadoop
Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install viirya-hadoop --version 0.0.18

Direct download is not typically how you would use a Puppet module to manage your infrastructure, but you may want to download the module in order to inspect the code.

Download
Tags: hadoop, yarn

Documentation

viirya/hadoop — version 0.0.18 May 28th 2014

Puppet module for deploying Hadoop MapReduce Next Generation on cluster

This module deploys Hadoop MapReduce Next Generation on a cluster of machines. It is tested on Apache Hadoop 2.2.0 under puppet agent/master environment. It is based on bcarpio/hadoop 0.0.3.

Usage:

Install this module on your puppet master node by:

sudo puppet install viirya-hadoop.

In site.pp, define:

node 'your hadoop slave nodes' {
    include java
    include hadoop::cluster::slave
}

node 'your hadoop master node' {
    include java
    include hadoop::cluster::master
}

For pseudo-distributed mode of Hadoop, put following codes in a .pp file (e.g. hadoop.pp).

include java
include hadoop::cluster::pseudomode

Then,

sudo puppet apply hadoop.pp

Hadoop Distribution:

This puppet module will automatically download Apache Hadoop distribution from one pre-defined Apache Mirror site. If you are like to use faster mirror site, please modify the URL in init.pp.

Parameters:

Some parameters are able to modify in params.pp. You should modify the parameters such as 'master', 'resourcemanager', 'dfs_slaves' and 'yarn_slaves' to reflect your hadoop cluster settings.

SSH keys:

Remember to generate your ssh keys and put the keys in files/ssh/.

Note: Since master deployment will run hadoop scripts to launch hadoop services on slaves nodes, please deploy hadoop slaves first. When all slaves are deployed, then deploy master node.

Kerberos:

Since 0.0.12 version, this puppet module supports Kerberos-secured Hadoop cluster. Kerberos should be installed first on your cluster. To generate necessary principles and keytab files for Hadoop serviers:

require hadoop::cluster::kerberos