storm

Module for managing Storm installation

deric

deric

81,076 downloads

6,063 latest version

4.6 quality score

Version information

  • 0.4.1 (latest)
  • 0.4.0
  • 0.3.1
  • 0.3.0
  • 0.2.0
  • 0.1.1
  • 0.1.0
released Oct 5th 2016
This version is compatible with:
  • RedHat
    ,
    CentOS
    ,
    Debian
    ,
    Ubuntu

Start using this module

Documentation

deric/storm — version 0.4.1 Oct 5th 2016

Puppet Storm

Puppet
Forge Build Status Puppet Forge
Downloads

Puppet module for managing Storm installation.

Features:

  • Hiera support
  • Mesos integration

Requirements

Usage

The binary package, which is by default called storm will be installed on all machines. You can provide a set of packages:

class {'storm':
  packages => ['storm', 'storm-mesos', 'libjzmq']
}

Main Storm class includes default configuration, each component is afterwards configured individually.

On master node include Nimbus:

class {'storm::nimbus': }

By default service management is disabled, to enable starting service with OS default init system (init.d/upstart/systemd/...) use parameter manage_service:

class {'storm::nimbus':
  manage_service => true
}

In order to change default service mechanism (determined by Puppet) use parameter force_provider:

class {'storm::nimbus':
  manage_service => true,
  force_provider => 'runit'
}

You can adjust all the parameters directly:

class {'storm::nimbus':
  host => '192.168.1.1'
}

or via Hiera:

storm::nimbus::host: '192.168.1.1'

Parameters:

  • manage_service by default false (service not managed by OS)
  • enable whether service should be enable (default: true, note manage_service must be also true)
  • force_provider default: undef
  • host address to bind, default: localhost
  • thrift_port default: 6627
  • childopts default -Xmx1024m
  • task_timeout_secs default 30
  • supervisor_timeout_secs default 60
  • monitor_freq_secs default 10
  • cleanup_inbox_freq_secs default 600
  • inbox_jar_expiration_secs default 3600
  • task_launch_secs default 120
  • reassign default true
  • file_copy_expiration_secs default 600 jvm array, default:
['-Dlog4j.configuration=file:/etc/storm/storm.log.properties','-Dlogfile.name=nimbus.log']

UI

class {'storm::ui': }

Parameters:

  • manage_service by default false (service not managed by OS)
  • enable whether service should be enable (default: true, note manage_service must be also true)
  • force_provider default: undef
  • mem JVM memory default 1024m
  • port default: 8080
  • childopts default -Xmx768m

Supervisor

Each computing node should include supervisor which watches Storm's bolts and spouts.

class {'storm::supervisor': }

DRPC

class {'storm::drpc': }

Mesos

Mesos integration requires special binary package which provides storm-mesos service (framework).

class {'storm::mesos': }

In Hiera you could define something like this:

storm::mesos::master_url: "%{mesos::zookeeper}"

Packages

Some configuration is shared between components, e.g. packages that are installed:

storm::packages: ['storm', 'libjzmq']

by default just storm package is installed.

Installation

puppet-librarian:

mod 'deric/storm', :git => 'https://github.com/deric/puppet-storm.git'

dependencies:

  • puppetlabs/stdlib
  • puppetlabs/concat