puppet-seed_stack

Puppet meta-module for deploying the Seed Stack. This module uses other Puppet modules to install all the basic parts of Praekelt Foundation's Seed Stack.

The module installs and configures:

• Mesos
• Marathon
• Zookeeper
• Docker
• Consul
• Consular
• Consul Template
• Nginx
• ...and a few miscellaneous things like Dnsmasq to tie them together

This module is not designed for complete configurability of the stack and instead takes an opinionated approach to setting up all the parts of the system to work together. This module should be the quickest way to set up a Seed Stack cluster (or even a standalone development host).

To try out Seed Stack, a Vagrant Box is available here.

This module was designed for use on Ubuntu 14.04.

Work in progress

This module is still a work in progress. We are waiting on new releases of a few of its dependencies. Some things are not done as nicely as they could be yet. There are no tests.

Usage

A Seed Stack node can either be a controller (i.e. a Mesos master), a worker (a Mesos slave), or a combination of the two.

The first step is to set up the cluster-wide parameters. A good way to do this is to define a class that all the nodes in the cluster can include. For example:

class my_cluster_params {
  class { 'seed_stack::cluster_params':
    advertise_addr   => $::ipaddress_eth1,
    controller_addrs => ['10.0.0.11', '10.0.0.12', '10.0.0.13'],
    consul_encrypt   => 'Oj4lWHxyQNGLkqH1L+7e4A==',
  }
}

node 'mynode.example.com' {
  include my_cluster_params
  ...
}

Setting up these cluster-wide parameters is optional but makes life easier. Every parameter can be set or overridden in the classes that use the parameters. At a minimum, most classes will require two parameters:

• advertise_addr: The address that other nodes in the cluster should use to access the node.
• controller_addrs: The list of advertise addresses for the controller nodes.

Controller

The seed_stack::controller class is responsible for configuring a Seed Stack controller. For a full list of available parameters, see the class source. If you've set up the cluster-wide parameters, this is simple.

node 'controller01.example.com', 'controller02.example.com', 'controller03.example.com' {
  include my_cluster_params
  include seed_stack::controller
}

Worker

The seed_stack::worker class is responsible for configuring a Seed Stack worker. For a full list of available parameters, see the class source. Again, after setting up the parameters you can do something like:

node /^worker\d{2}\.example\.com$/ {
  include my_cluster_params
  include seed_stack::worker
}

Combination controller/worker

A node can be both a controller and a worker. This is useful for single-node setups.

node 'standalone.example.com' {
  class { 'seed_stack::cluster_params':
    advertise_addr    => $ipaddress_eth0,
    controller_addrs  => [$ipaddress_eth0],
  }
  class { 'seed_stack::controller':
    controller_worker => true,
  }
  class { 'seed_stack::worker':
    controller_worker => true,
  }
}

NOTE: For combination controller/workers it is necessary to set controller_worker => true for both the seed_stack::controller class and the seed_stack::worker class so that the two classes do not conflict.

External load balancers and routers

We use Nginx to load balance and route between containers. Sometimes it is useful to do this outside of the Mesos cluster itself. For instance, an external host could be the load balancer for a web service or some service running on the host could need to be routed to containers directly.

Nginx is dynamically configured using Consul Template in these cases. First, set that up:

class { 'seed_stack::template_nginx':
  consul_address => '192.168.0.5', # Consul address for Consul Template to connect to
}

Then you can set up either set up a load balancer or a router (or both):

include seed_stack::load_balancer
include seed_stack::router

Consul DNS

It's often useful to use Consul's DNS for service discovery on nodes that aren't controllers or workers. For example, a database node could advertise it's service to other nodes using Consul. To do this, use the seed_stack::consul_dns class. The class needs a few parameters so that it knows how to join the Consul cluster:

class { 'seed_stack::consul_dns':
  advertise_addr => $ipaddress_eth0, # Address to advertise for services on this node
  join           => ['10.215.32.11', '10.215.32.12'], # List of any Consul nodes already in the cluster
}

consul::service { 'postgresql':
  port   => 5432,
  checks => [{
    script   => '/usr/bin/pg_isready',
    interval => '30s',
  }],
}

After the above example is applied, the address postgresql.service.consul is available in the Consul cluster and will point to the node's advertise address. For full documentation on all the configuration parameters available for Consul, see the manifest source.

Upstream modules

We make use of quite a few Puppet modules to manage all the various pieces of software that make up Seed Stack. See the metadata file for a complete listing with version information.

Firstly, we wrote some modules ourselves:

• praekeltfoundation/consular
• praekeltfoundation/marathon
• praekeltfoundation/webupd8_oracle_java

Then there are some 3rd party modules:

• deric/mesos
• deric/zookeeper
• garethr/docker
• gdhbashton/consul_template
• KyleAnderson/consul

Java 8

Java 8 is a dependency of Marathon version 0.12.0+. Ubuntu 14.04 does not have a package for this in the standard repositories. seed_stack::controller will install Oracle Java 8 from the WebUpd8 PPA. This can be disabled by passing install_java => false.

Note that Java is also a dependency of Mesos and Zookeeper. If you are managing your own Java installation you should ensure that Java is installed before any of the packages that depend on it so as to prevent multiple versions of Java being installed.

Default package versions

The package versions can be seen in the params class source. All of these versions can be adjusted using parameters in the controller and worker classes. These versions are reasonably well-tested and known to work together.

0.7.0 - 2016/03/07

Features

• New seed_stack::cluster_params class makes it easier to set cluster-wide configuration parameters (#74)

Changes

• advertise_addr and join are now required parameters of seed_stack::consul_dns (#74)

Fixes

• Install Mesos package with the --no-install-recommends flag when using versions of Puppet >= 3.6.0. This should ensure that Zookeeper is not installed unnecessarily. With older versions of Puppet, stop Zookeeper from running on workers. (#76)
• Include gluster::client on worker hosts by default. (#79)

0.6.2 - 2016/03/03

Changes

• Package versions now pinned to full version strings - NOTE: versions now platform specific (#78)

Fixes

• Changed Dnsmasq servicehost entry from address to host-record (#73)
• Pin package versions to complete version strings to prevent service restarts (#77)

0.6.1 - 2016/03/02

Fixes

• Fix Consul DNS lookup via Dnsmasq in certain cases (#72)

0.6.0 - 2016/02/24

Features

• Add redis_host option for Xylem (#68)

0.5.0 - 2016/02/23

Features

• Initial xylem support (#65)
• Docker 1.10.2 (#66)
• Mesos 0.27.1 (#66)
• Marathon 0.15.3 (badf8e9)
• Consul Template 0.13.0 (#63)

Changes

• Consul working directory is now /var/lib/consul (#58)
• deric/zookeeper Puppet module updated to 0.5.1 (#64)

Fixes

• Some basic tests (#57, #3, #65)

0.4.0 - 2016/02/09

Features

• Dnsmasq host alias (servicehost) on all workers and controllers (#25, #46)
• Nginx service router address, port and domain now configurable (#44)
• Docker 1.10.0 (#47)
• Mesos 0.27.0 (#35)
• Marathon 0.15.1 (#51)
• Marathon syslog output is now suppressed (#54)
• README improvements (#48)
• Setting up a cluster now only requires the controllers being able to resolve the hostnames of workers (no other lookups are required) (#35)

Changes

• controller_addresses -> controller_addrs, address -> advertise_addr (#50)
• controller_addrs and advertise_addr are now required parameters for controllers and workers (#49)
• Host advertise_addr used as Docker DNS address which means things should provision in a single Puppet run (#43)
• deric/mesos Puppet module updated to 0.6.5 (#40)
• praekeltfoundation/marathon Puppet module updated to 0.4.0 (#54)

Fixes

• Nginx service router template properly parses internal boolean value (#44)
• Worker hostname defaults to $::fqdn like controllers (#35)
• Nginx service router actually works now (#25, #35, #44, #45)

0.3.0 - 2016/01/20

Features

• seed_stack::consul_dns class makes it easy to set up Consul with DNS on a node outside the Mesos cluster (#32)
• Marathon 0.14.0 and Mesos 0.26.0 (#34)

Changes

• Simplified Mesos service management (#27)
• Changes to the way seed_stack::load_balancer is structured that affect configuration (#28)
• praekeltfoundation/consular 0.2.0 module (#29)
• Consul 0.6.3 and Consul Template 0.12.2 (#30)
• Greater use of Puppet stdlib functions instead of inline_template (#33)

Fixes

• Dnsmasq should now work when Consul listen address is not localhost (#26)
• Consular should now work when the Consul listen address is not localhost (#29)
• Unzip should now definitely be installed before installing Consul or Consul Template (#32, #36)

0.2.3 - 2016/01/15

Changes

• deric/zookeeper module version 0.4.2 - Zookeeper maxClientCnxns is no longer set. The default is used. This should have no effect as we were previously setting it to the default. (#21)
• Consul Template 0.12.1 (#22)
• Default the hostname parameter to $::fqdn. This makes it easier for Marathon/Mesos/Nginx to look up other nodes, especially on a public network (although you probably shouldn't be doing that). (#24)

Fixes

• Fix worker Dnsmasq configuration (6088bb6)

0.2.2 - 2016/01/14

Changes

• Mesos master advertise_ip now set to address (#19)

Fixes

• Marathon module updated to 0.2.2 with fixes (#23)

0.2.1 - 2016/01/12

Changes

• Source praekeltfoundation modules from Puppet Forge (#10)

0.2.0 - 2016/01/12

Features

• seed_stack::load_balancer added for exposing services externally (#11)
• Mesos listen address now configurable (#13)
• Consul health checks for Mesos, Marathon and Zookeeper (#9)
• Zookeeper listen address now configurable (#17)

Changes

• Node address and hostname default to $::ipaddress_lo and $::hostname, respectively. (#14)
• Mesos listen address defaults to 0.0.0.0 (#13)
• Zookeeper listen address defaults to 0.0.0.0 (#17)
• Combination controller/worker nodes must now set controller_worker => true for both seed_stack::controller and seed_stack::worker classes. (#15)
• Use git to fetch praekeltfoundation modules (#16)
• Zookeeper maxClientCnxns now set to 60, up from 10 (#17)

Fixes

• Ensure apt-get has updated before installing Mesos on a worker (#12)
• Fix installing Consul on a worker due to missing unzip package (#12)
• Stop mesos-slave service from running on controller and mesos-master from running on worker (#15)
• Add mesos-slave Consul service to combination controller/worker (#15)
• Fix Consul on workers by removing bootstrap_expect (#15)
• Determine and set Zookeeper ID correctly (#17)

0.1.0 - 2016/01/08

Initial release

• seed_stack::controller to set up a controller
• seed_stack::worker to set up a worker
• Tested on a standalone combination controller/worker only