Install Hadoop MapReduce Next Generation.




9,335 latest version

1.5 quality score

Version information

  • 0.0.18 (latest)
  • 0.0.17
  • 0.0.16
  • 0.0.15
  • 0.0.14
  • 0.0.13
  • 0.0.12
  • 0.0.11
  • 0.0.10
  • 0.0.9
  • 0.0.8
  • 0.0.7
  • 0.0.6
  • 0.0.5
  • 0.0.4
  • 0.0.3
  • 0.0.2
  • 0.0.1
released May 28th 2014

Start using this module

Tags: hadoop, yarn


viirya/hadoop — version 0.0.18 May 28th 2014

Puppet module for deploying Hadoop MapReduce Next Generation on cluster

This module deploys Hadoop MapReduce Next Generation on a cluster of machines. It is tested on Apache Hadoop 2.2.0 under puppet agent/master environment. It is based on bcarpio/hadoop 0.0.3.


Install this module on your puppet master node by:

sudo puppet install viirya-hadoop.

In site.pp, define:

node 'your hadoop slave nodes' {
    include java
    include hadoop::cluster::slave

node 'your hadoop master node' {
    include java
    include hadoop::cluster::master

For pseudo-distributed mode of Hadoop, put following codes in a .pp file (e.g. hadoop.pp).

include java
include hadoop::cluster::pseudomode


sudo puppet apply hadoop.pp

Hadoop Distribution:

This puppet module will automatically download Apache Hadoop distribution from one pre-defined Apache Mirror site. If you are like to use faster mirror site, please modify the URL in init.pp.


Some parameters are able to modify in params.pp. You should modify the parameters such as 'master', 'resourcemanager', 'dfs_slaves' and 'yarn_slaves' to reflect your hadoop cluster settings.

SSH keys:

Remember to generate your ssh keys and put the keys in files/ssh/.

Note: Since master deployment will run hadoop scripts to launch hadoop services on slaves nodes, please deploy hadoop slaves first. When all slaves are deployed, then deploy master node.


Since 0.0.12 version, this puppet module supports Kerberos-secured Hadoop cluster. Kerberos should be installed first on your cluster. To generate necessary principles and keytab files for Hadoop serviers:

require hadoop::cluster::kerberos