Forge Home

zookeeper

Zookeeper cluster with security

18,974 downloads

1,164 latest version

5.0 quality score

We run a couple of automated
scans to help you access a
module's quality. Each module is
given a score based on how well
the author has formatted their
code and documentation and
modules are also checked for
malware using VirusTotal.

Please note, the information below
is for guidance only and neither of
these methods should be considered
an endorsement by Puppet.

Version information

  • 1.0.2 (latest)
  • 1.0.1
  • 1.0.0
  • 0.12.0
  • 0.11.0
  • 0.10.0
  • 0.9.9
  • 0.9.8
  • 0.9.7
  • 0.9.6
  • 0.9.5
  • 0.9.4
  • 0.9.3
  • 0.9.2
  • 0.9.1
released Sep 26th 2021
This version is compatible with:
  • Puppet Enterprise 2023.8.x, 2023.7.x, 2023.6.x, 2023.5.x, 2023.4.x, 2023.3.x, 2023.2.x, 2023.1.x, 2023.0.x, 2021.7.x, 2021.6.x, 2021.5.x, 2021.4.x, 2021.3.x, 2021.2.x, 2021.1.x, 2021.0.x, 2019.8.x, 2019.7.x, 2019.5.x, 2019.4.x, 2019.3.x, 2019.2.x, 2019.1.x, 2019.0.x, 2018.1.x, 2017.3.x, 2017.2.x, 2017.1.x, 2016.5.x, 2016.4.x
  • Puppet >= 3.4.0
  • , , , ,

Start using this module

  • r10k or Code Manager
  • Bolt
  • Manual installation
  • Direct download

Add this module to your Puppetfile:

mod 'cesnet-zookeeper', '1.0.2'
Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add cesnet-zookeeper
Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install cesnet-zookeeper --version 1.0.2

Direct download is not typically how you would use a Puppet module to manage your infrastructure, but you may want to download the module in order to inspect the code.

Download

Documentation

cesnet/zookeeper — version 1.0.2 Sep 26th 2021

Apache Zookeeper Puppet Module

Build Status Puppet Forge

####Table of Contents

  1. Module Description - What the module does and why it is useful
  2. Setup - The basics of getting started with Zookeeper
  3. Usage - Configuration options and additional functionality
  4. Reference - An under-the-hood peek at what the module is doing and how
  5. Development - Guide for contributing to the module

##Module Description

This module installs and configures Apache Zookeeper quorum cluster. It expects list of hostnames, where zookeeper should be running. Zookeeper IDs will be generated according to the ordering of these hostnames.

Optionally the security based on Kerberos can be enabled.

Tested on:

  • Debian 7/wheezy: Cloudera distribution (tested with CDH 5.3.0/5.5.1/5.7.1, Zookeeper 3.4.5)
  • Debian 8/jessie: BigTop distribution (BigTop 1.2.0, Zookeeper 3.4.6)
  • Ubuntu 14/trusty
  • RHEL 6/7 and clones: Cloudera distribution (tested with CDH 5.4.2, Zookeeper 3.4.5)

##Setup

###What cesnet-zookeeper module affects

  • Packages: zookeeper server package
  • Alternatives:
  • alternatives are used for /etc/zookeeper/conf in Cloudera
  • this module switches to the new alternative by default on Debian, so the Cloudera original configuration can be kept intact
  • Files modified:
  • */etc/zookeeper/conf**
  • */var/lib/zookeeper/**
  • Secret files (keytab): ownerships and permissions modified
  • Java system properties set for Zookeeper:
  • java.security.auth.login.config
  • zookeeper.security.auth_to_local

###Setup Requirements

There are several known or intended limitations in this module.

Be aware of:

  • Repositories - see cesnet-hadoop module Setup Requirements for details

  • Secure mode: keytab must be prepared in /etc/security/keytabs/zookeeper.service.keytab (see realm parameter)

###Beginning with Zookeeper

Example: one-machine zookeeper quorum without security:

class{'zookeeper':
  hostnames => [ $::fqdn ],
}
include ::zookeeper::server

It is recommended to have at least three or more (odd-numbered) zookeeper machines. All zookeeper hostnames must be specified in hostnames and the order must be the same across all the nodes.

##Usage

Example: Setup with security:

class{'zookeeper':
  hostnames => [ $::fqdn ],
  realm     => 'MY.REALM',
}
include ::zookeeper::server

The keytab file must be available at /etc/security/keytabs/zookeeper.service.keytab.

Note: you can consider removing or changing property zookeeper.security.auth_to_local:

properties => {
  'zookeeper.security.auth_to_local' => '::undef',
}

Default value is valid for principal names according to Hadoop documentation at http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SecureMode.html and it is needed only with cross-realm authentication.

###Superuser Access

It is recommended to set super user credentials (for example to be able to restore bad ACLs).

####Get the digest string: (replace $PASSWORD by real password)

export ZK_HOME=/usr/lib/zookeeper
java -cp $ZK_HOME/lib/*:$ZK_HOME/zookeeper.jar org.apache.zookeeper.server.auth.DigestAuthenticationProvider super:$PASSWORD

####Use the digest in properties:

class{'zookeeper':
  hostnames  => [ $::fqdn ],
  realm      => 'MY.REALM',
  properties => {
    zookeeper.DigestAuthenticationProvider.superDigest => 'super:XXXXX',
  },
}
include ::zookeeper::server

####Using in the client:

zooclient-cli
  addauth digest super:PASSWORD

##Reference

###Classes

  • zookeeper: Configuration class for Zookeeper
  • zookeeper::client: Zookeeper client
  • zookeeper::client::config: Zookeeper client configuration
  • zookeeper::client::install: Zookeeper client installation
  • zookeeper::client::service: Stub class
  • zookeeper::common::config: Zookeeper common configuration
  • zookeeper::common::postinstall: Preparation steps after installation
  • zookeeper::params: Zookeeper module parameters
  • zookeeper::server: Zookeeper node
  • zookeeper::server::config: Zookeeper server configuration
  • zookeeper::server::install: Zookeeper node installation
  • zookeeper::server::service: Launch zookeeper service

###Module Parameters

####alternatives

Switches the alternatives used for the configuration. Default: 'cluster' (Debian) or undef.

It can be used only when supported (for example with Cloudera distribution).

####hostnames

Array of zookeeper nodes hostnames. Default: undef.

####myid

ID of zookeeper server in the quorum. Default: undef (=autodetect).

myid is the ID number of the zookeeper server in the quorum. It's the number starting from 1 and it must be unique for each node.

By default, the ID is generated automatically as order of the node hostname (::fqdn) in the hostnames array.

####properties

Generic properties to be set for the zookeeper cluster. Default: undef.

Some properties are set automatically, "::undef" string explicitly removes given property. Empty string sets the empty value.

####keytab

Server keytab file. Default: '/etc/security/keytab/zookeeper.service.keytab'.

####principal

Server principal. Default: "zookeeper/${::fqdn}@${realm}".

####realm

Enables security and specifies Kerberos realm to use. Default: ''.

Empty string disables the security.

With enabled security there are required:

  • configured Kerberos (/etc/krb5.conf)
  • /etc/security/keytab/zookeeper.service.keytab (on zookeeper nodes)

##Development