Elasticsearch Puppet Module

Table of Contents

1. Module description - What the module does and why it is useful
2. Setup - The basics of getting started with Elasticsearch

• The module manages the following
• Requirements

3. Usage - Configuration options and additional functionality
4. Advanced features - Extra information on advanced usage
5. Reference - An under-the-hood peek at what the module is doing and how
6. Limitations - OS compatibility, etc.
7. Development - Guide for contributing to the module
8. Support - When you need help with this module

Module description

This module sets up Elasticsearch instances with additional resource for plugins, templates, and more.

This module has been tested against all versions of ES 1.x, 2.x, and 5.x.

Setup

The module manages the following

• Elasticsearch repository files.
• Elasticsearch package.
• Elasticsearch configuration file.
• Elasticsearch service.
• Elasticsearch plugins.
• Elasticsearch templates.
• Elasticsearch Shield users, roles, and certificates.

Requirements

• The stdlib Puppet library.
• ceritsc/yum For yum version lock.
• richardc/datacat
• Augeas
• puppetlabs-java for Java installation (optional).
• puppetlabs-java_ks for Shield certificate management (optional).

Repository management

When using the repository management, the following module dependencies are required:

• Debian/Ubuntu: Puppetlabs/apt
• OpenSuSE/SLES: Darin/zypprepo

Beginning with Elasticsearch

Declare the top-level elasticsearch class (managing repositories) and set up an instance:

class { 'elasticsearch':
  java_install => true,
  manage_repo  => true,
  repo_version => '5.x',
}

elasticsearch::instance { 'es-01': }

Note: Elasticsearch 5.x requires a recent version of the JVM. If you are on a recent version of your distribution of choice (such as Ubuntu 16.04 or CentOS 7), setting java_install => true will work out-of-the-box. If you are on an earlier distribution, you may need to take additional measures to install Java 1.8.

Usage

Main class

Most top-level parameters in the elasticsearch class are set to reasonable defaults. The following are some parameters that may be useful to override:

Install a specific version

class { 'elasticsearch':
  version => '1.4.2'
}

Note: This will only work when using the repository.

Automatically restarting the service (default set to false)

By default, the module will not restart Elasticsearch when the configuration file, package, or plugins change. This can be overridden globally with the following option:

class { 'elasticsearch':
  restart_on_change => true
}

Or controlled with the more granular options: restart_config_change, restart_package_change, and restart_plugin_change.

Automatic upgrades (default set to false)

class { 'elasticsearch':
  autoupgrade => true
}

Removal/Decommissioning

class { 'elasticsearch':
  ensure => 'absent'
}

Install everything but disable service(s) afterwards

class { 'elasticsearch':
  status => 'disabled'
}

API Settings

Some resources, such as elasticsearch::template, require communicating with the Elasticsearch REST API. By default, these API settings are set to:

class { 'elasticsearch':
  api_protocol            => 'http',
  api_host                => 'localhost',
  api_port                => 9200,
  api_timeout             => 10,
  api_basic_auth_username => undef,
  api_basic_auth_password => undef,
  api_ca_file             => undef,
  api_ca_path             => undef,
  validate_tls            => true,
}

Each of these can be set at the top-level elasticsearch class and inherited for each resource or overridden on a per-resource basis.

Instances

This module works with the concept of instances. For service to start you need to specify at least one instance.

Quick setup

elasticsearch::instance { 'es-01': }

This will set up its own data directory and set the node name to $hostname-$instance_name

Advanced options

Instance specific options can be given:

elasticsearch::instance { 'es-01':
  config        => { }, # Configuration hash
  init_defaults => { }, # Init defaults hash
  datadir       => [ ], # Data directory
}

See Advanced features for more information.

Plugins

This module can help manage a variety of plugins. Note that module_dir is where the plugin will install itself to and must match that published by the plugin author; it is not where you would like to install it yourself.

From an official repository

elasticsearch::plugin { 'lmenezes/elasticsearch-kopf':
  instances => 'instance_name'
}

From a custom url

elasticsearch::plugin { 'jetty':
  url        => 'https://oss-es-plugins.s3.amazonaws.com/elasticsearch-jetty/elasticsearch-jetty-1.2.1.zip',
  instances  => 'instance_name'
}

Using a proxy

You can also use a proxy if required by setting the proxy_host and proxy_port options:

elasticsearch::plugin { 'lmenezes/elasticsearch-kopf',
  instances  => 'instance_name',
  proxy_host => 'proxy.host.com',
  proxy_port => 3128
}

Proxies that require usernames and passwords are similarly supported with the proxy_username and proxy_password parameters.

Plugin name formats that are supported include:

• elasticsearch/plugin/version (for official elasticsearch plugins downloaded from download.elastic.co)
• groupId/artifactId/version (for community plugins downloaded from maven central or OSS Sonatype)
• username/repository (for site plugins downloaded from github master)

Upgrading plugins

When you specify a certain plugin version, you can upgrade that plugin by specifying the new version.

elasticsearch::plugin { 'elasticsearch/elasticsearch-cloud-aws/2.1.1': }

And to upgrade, you would simply change it to

elasticsearch::plugin { 'elasticsearch/elasticsearch-cloud-aws/2.4.1': }

Please note that this does not work when you specify 'latest' as a version number.

ES 2.x official plugins

For the Elasticsearch commercial plugins you can refer them to the simple name.

See Plugin installation for more details.

Scripts

Installs scripts to be used by Elasticsearch. These scripts are shared across all defined instances on the same host.

elasticsearch::script { 'myscript':
  ensure => 'present',
  source => 'puppet:///path/to/my/script.groovy'
}

Templates

By default templates use the top-level elasticsearch::api_* settings to communicate with Elasticsearch. The following is an example of how to override these settings:

elasticsearch::template { 'templatename':
  api_protocol            => 'https',
  api_host                => $::ipaddress,
  api_port                => 9201,
  api_timeout             => 60,
  api_basic_auth_username => 'admin',
  api_basic_auth_password => 'adminpassword',
  api_ca_file             => '/etc/ssl/certs',
  api_ca_path             => '/etc/pki/certs',
  validate_tls            => false,
  source                  => 'puppet:///path/to/template.json',
}

Add a new template using a file

This will install and/or replace the template in Elasticsearch:

elasticsearch::template { 'templatename':
  source => 'puppet:///path/to/template.json',
}

Add a new template using content

This will install and/or replace the template in Elasticsearch:

elasticsearch::template { 'templatename':
  content => {
    'template' => "*",
    'settings' => {
      'number_of_replicas' => 0
    }
  }
}

Plain JSON strings are also supported.

elasticsearch::template { 'templatename':
  content => '{"template":"*","settings":{"number_of_replicas":0}}'
}

Delete a template

elasticsearch::template { 'templatename':
  ensure => 'absent'
}

Bindings/Clients

Install a variety of clients/bindings:

Python

elasticsearch::python { 'rawes': }

Ruby

elasticsearch::ruby { 'elasticsearch': }

Connection Validator

This module offers a way to make sure an instance has been started and is up and running before doing a next action. This is done via the use of the es_instance_conn_validator resource.

es_instance_conn_validator { 'myinstance' :
  server => 'es.example.com',
  port   => '9200',
}

A common use would be for example :

class { 'kibana4' :
  require => Es_Instance_Conn_Validator['myinstance'],
}

Package installation

There are two different ways of installing Elasticsearch:

Repository

This option allows you to use an existing repository for package installation. The repo_version corresponds with the major.minor version of Elasticsearch for versions before 2.x.

class { 'elasticsearch':
  manage_repo  => true,
  repo_version => '1.4',
}

For 2.x versions of Elasticsearch, use repo_version => '2.x'.

class { 'elasticsearch':
  manage_repo  => true,
  repo_version => '2.x',
}

Remote package source

When a repository is not available or preferred you can install the packages from a remote source:

http/https/ftp

class { 'elasticsearch':
  package_url => 'https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.4.2.deb',
  proxy_url   => 'http://proxy.example.com:8080/',
}

Setting proxy_url to a location will enable download using the provided proxy server. This parameter is also used by elasticsearch::plugin. Setting the port in the proxy_url is mandatory. proxy_url defaults to undef (proxy disabled).

puppet://

class { 'elasticsearch':
  package_url => 'puppet:///path/to/elasticsearch-1.4.2.deb'
}

Local file

class { 'elasticsearch':
  package_url => 'file:/path/to/elasticsearch-1.4.2.deb'
}

Java installation

Most sites will manage Java separately; however, this module can attempt to install Java as well. This is done by using the puppetlabs-java module.

class { 'elasticsearch':
  java_install => true
}

Specify a particular Java package/version to be installed:

class { 'elasticsearch':
  java_install => true,
  java_package => 'packagename'
}

When configuring Elasticsearch's memory usage, you can do so by either changing init defaults for Elasticsearch 1.x/2.x (see the following example), or modify it globally in 5.x using jvm.options:

class { 'elasticsearch':
  jvm_options => [
    '-Xms4g',
    '-Xmx4g'
  ]
}

Service management

Currently only the basic SysV-style init and Systemd service providers are supported, but other systems could be implemented as necessary (pull requests welcome).

Defaults File

The defaults file (/etc/defaults/elasticsearch or /etc/sysconfig/elasticsearch) for the Elasticsearch service can be populated as necessary. This can either be a static file resource or a simple key value-style hash object, the latter being particularly well-suited to pulling out of a data source such as Hiera.

File source

class { 'elasticsearch':
  init_defaults_file => 'puppet:///path/to/defaults'
}

Hash representation

$config_hash = {
  'ES_HEAP_SIZE' => '30g',
}

class { 'elasticsearch':
  init_defaults => $config_hash
}

Note: init_defaults hash can be passed to the main class and to the instance.

Advanced features

Shield

Shield users, roles, and certificates can be managed by this module.

Note: If you are planning to use these features, it is highly recommended you read the following documentation to understand the caveats and extent of the resources available to you.

Getting Started

Although this module can handle several types of Shield resources, you are expected to manage the plugin installation and versions for your deployment. For example, the following manifest will install Elasticseach with a single instance running shield:

class { 'elasticsearch':
  java_install => true,
  manage_repo  => true,
  repo_version => '1.7',
}

elasticsearch::instance { 'es-01': }

Elasticsearch::Plugin { instances => ['es-01'], }
elasticsearch::plugin { 'elasticsearch/license/latest': }
elasticsearch::plugin { 'elasticsearch/shield/latest': }

The following examples will assume the preceding resources are part of your puppet manifest.

Roles

Roles in the esusers realm can be managed using the elasticsearch::shield::role type. For example, to create a role called myrole, you could use the following resource:

elasticsearch::shield::role { 'myrole':
  privileges => {
    'cluster' => 'monitor',
    'indices' => {
      '*' => 'read'
    }
  }
}

This role would grant users access to cluster monitoring and read access to all indices. See the Shield documentation for your version to determine what privileges to use and how to format them (the Puppet hash representation will simply be translated into yaml.)

Note: The Puppet provider for esusers has fine-grained control over the roles.yml file and thus will leave the default roles Shield installs in-place. If you would like to explicitly purge the default roles (leaving only roles managed by puppet), you can do so by including the following in your manifest:

resources { 'elasticsearch_shield_role':
  purge => true,
}

Mappings

Associating mappings with a role is done by passing an array of strings to the mappings parameter of the elasticsearch::shield::role type. For example, to define a role with mappings using Shield >= 2.3.x style role definitions:

elasticsearch::shield::role { 'logstash':
  mappings   => [
    'cn=group,ou=devteam',
  ],
  privileges => {
    'cluster' => 'manage_index_templates',
    'indices' => [{
      'names'      => ['logstash-*'],
      'privileges' => [
        'write',
        'delete',
        'create_index',
      ],
    }],
  },
}

Note: Observe the brackets around indices in the preceding role definition; which is an array of hashes per the format in Shield 2.3.x. Follow the documentation to determine the correct formatting for your version of Shield.

If you'd like to keep the mappings file purged of entries not under Puppet's control, you should use the following resources declaration because mappings are a separate low-level type:

resources { 'elasticsearch_shield_role_mapping':
  purge => true,
}

Users

Users can be managed using the elasticsearch::shield::user type. For example, to create a user mysuser with membership in myrole:

elasticsearch::shield::user { 'myuser':
  password => 'mypassword',
  roles    => ['myrole'],
}

The password parameter will also accept password hashes generated from the esusers utility and ensure the password is kept in-sync with the Shield users file for all Elasticsearch instances.

elasticsearch::shield::user { 'myuser':
  password => '$2a$10$IZMnq6DF4DtQ9c4sVovgDubCbdeH62XncmcyD1sZ4WClzFuAdqspy',
  roles    => ['myrole'],
}

Note: When using the esusers provider (the default for plaintext passwords), Puppet has no way to determine whether the given password is in-sync with the password hashed by Shield. In order to work around this, the elasticsearch::shield::user resource has been designed to accept refresh events in order to update password values. This is not ideal, but allows you to instruct the resource to change the password when needed. For example, to update the aforementioned user's password, you could include the following your manifest:

notify { 'update password': } ~>
elasticsearch::shield::user { 'myuser':
  password => 'mynewpassword',
  roles    => ['myrole'],
}

Certificates

SSL/TLS can be enabled by providing an elasticsearch::instance type with paths to the certificate and private key files, and a password for the keystore.

elasticsearch::instance { 'es-01':
  ssl                  => true,
  ca_certificate       => '/path/to/ca.pem',
  certificate          => '/path/to/cert.pem',
  private_key          => '/path/to/key.pem',
  keystore_password    => 'keystorepassword',
}

Note: Setting up a proper CA and certificate infrastructure is outside the scope of this documentation, see the aforementioned Shield guide for more information regarding the generation of these certificate files.

The module will set up a keystore file for the node to use and set the relevant options in elasticsearch.yml to enable TLS/SSL using the certificates and key provided.

System Keys

Shield system keys can be passed to the module, where they will be placed into individual instance configuration directories. This can be set at the elasticsearch class and inherited across all instances:

class { 'elasticsearch':
  system_key => 'puppet:///path/to/key',
}

Or set on a per-instance basis:

elasticsearch::instance { 'es-01':
  system_key => '/local/path/to/key',
}

Package version pinning

The module supports pinning the package version to avoid accidental upgrades that are not done by Puppet. To enable this feature:

class { 'elasticsearch':
  package_pin => true,
  version     => '1.5.2',
}

In this example we pin the package version to 1.5.2.

Data directories

There are 4 different ways of setting data directories for Elasticsearch. In every case the required configuration options are placed in the elasticsearch.yml file.

Default

By default we use:

/usr/share/elasticsearch/data/$instance_name

Which provides a data directory per instance.

Single global data directory

class { 'elasticsearch':
  datadir => '/var/lib/elasticsearch-data'
}

Creates the following for each instance:

/var/lib/elasticsearch-data/$instance_name

Multiple Global data directories

class { 'elasticsearch':
  datadir => [ '/var/lib/es-data1', '/var/lib/es-data2']
}

Creates the following for each instance: /var/lib/es-data1/$instance_name and /var/lib/es-data2/$instance_name.

Single instance data directory

class { 'elasticsearch': }

elasticsearch::instance { 'es-01':
  datadir => '/var/lib/es-data-es01'
}

Creates the following for this instance:

/var/lib/es-data-es01

Multiple instance data directories

class { 'elasticsearch': }

elasticsearch::instance { 'es-01':
  datadir => ['/var/lib/es-data1-es01', '/var/lib/es-data2-es01']
}

Creates the following for this instance: /var/lib/es-data1-es01 and /var/lib/es-data2-es01.

Main and instance configurations

The config option in both the main class and the instances can be configured to work together.

The options in the instance config hash will merged with the ones from the main class and override any duplicates.

Simple merging

class { 'elasticsearch':
  config => { 'cluster.name' => 'clustername' }
}

elasticsearch::instance { 'es-01':
  config => { 'node.name' => 'nodename' }
}
elasticsearch::instance { 'es-02':
  config => { 'node.name' => 'nodename2' }
}

This example merges the cluster.name together with the node.name option.

Overriding

When duplicate options are provided, the option in the instance config overrides the ones from the main class.

class { 'elasticsearch':
  config => { 'cluster.name' => 'clustername' }
}

elasticsearch::instance { 'es-01':
  config => { 'node.name' => 'nodename', 'cluster.name' => 'otherclustername' }
}

elasticsearch::instance { 'es-02':
  config => { 'node.name' => 'nodename2' }
}

This will set the cluster name to otherclustername for the instance es-01 but will keep it to clustername for instance es-02

Configuration writeup

The config hash can be written in 2 different ways:

Full hash writeup

Instead of writing the full hash representation:

class { 'elasticsearch':
  config                 => {
   'cluster'             => {
     'name'              => 'ClusterName',
     'routing'           => {
        'allocation'     => {
          'awareness'    => {
            'attributes' => 'rack'
          }
        }
      }
    }
  }
}

Short hash writeup

class { 'elasticsearch':
  config => {
    'cluster' => {
      'name' => 'ClusterName',
      'routing.allocation.awareness.attributes' => 'rack'
    }
  }
}

Limitations

This module is built upon and tested against the versions of Puppet listed in the metadata.json file (i.e. the listed compatible versions on the Puppet Forge).

The module has been tested on:

• Debian 7/8
• CentOS 6/7
• OracleLinux 6/7
• Ubuntu 12.04, 14.04, 16.04
• OpenSuSE 42.x
• SLES 12

Other distro's that have been reported to work:

• RHEL 6
• Scientific 6

Testing on other platforms has been light and cannot be guaranteed.

Development

Please see the CONTRIBUTING.md file for instructions regarding development environments and testing.

Support

Need help? Join us in #elasticsearch on Freenode IRC or on the discussion forum.

0.16.2 (February 22, 2017)

Clarified supported versions in the README and added official deprecation notice to the module summary.

0.16.1 (February 9, 2017)

Summary

With this release, the 0.x series of the module is deprecated. Future updates will be released under the Puppet Forge namespace elastic/elasticsearch. Note that upgrading from this version is a major release and the update documentation should be reviewed for breaking changes.

0.16.0 (January 26, 2017)

Summary

Bugfix release including support for configuring the jvm.options file.

Features

• Support management of the global jvm.options configuration file.
• Restricted permissions to the elasticsearch.yml file.

Bugfixes

Changes

• sysctl settings are no longer managed by the thias/sysctl module.
• Calls to elasticsearch -version in elasticsearch::plugin code replaced with native Puppet code to resolve Elasticsearch package version. Should improve resiliency when managing plugins.

Testing changes

0.15.1 (December 1, 2016)

Summary

Primarily a bugfix release for Elasticsearch 5.x support-related issues. Note updated minimum required puppet versions as well.

Features

Bugfixes

• Removed ES_HEAP_SIZE check in init scripts for Elasticsearch 5.x
• Changed sysctl value to a string to avoid type errors for some versions
• Fixed a $LOAD_PATH error that appeared in some cases for puppet_x/elastic/es_versioning

Changes

• Updated minimium required version for Puppet and PE to reflect tested versions and versions supported by Puppet Labs

Testing changes

0.15.0 (November 17, 2016)

Summary

• Support for Ubuntu Xenial (16.04) formally declared.
• Initial support for running Elasticsearch 5.x series.

Features

• Support management of 5.x-style Elastic yum/apt package repositories.
• Support service scripts for 5.x series of Elasticsearch

Bugfixes

• Update the apt::source call to not cause deprecation warnings
• Updated module metadata to correctly require puppet-stdlib with validate_integer()

Changes

Testing changes

• Ubuntu Xenial (16.04) added to the test matrix.

0.14.0 (October 12, 2016)

Summary

Primarily a bugfix release for issues related to plugin proxy functionality, various system service fixes, and directory permissions. This release also adds the ability to define logging rolling file settings and a CA file/path for template API access.

Features

• Added 'file_rolling_type' parameter to allow selecting file logging rotation type between "dailyRollingFile" or "rollingFile". Also added 'daily_rolling_date_pattern', 'rolling_file_max_backup_index' and 'rolling_file_max_file_size' for file rolling customization.

Bugfixes

• Permissions on the Elasticsearch plugin directory have been fixed to permit world read rights.
• The service systemd unit now Wants= a network target to fix bootup parallelization problems.
• Recursively create the logdir for elasticsearch when creating multiple instances
• Files and directories with root ownership now specify UID/GID 0 instead to improve compatability with *BSDs.
• Elasticsearch Debian init file changed to avoid throwing errors when DATA_DIR, WORK_DIR and/or LOG_DIR were an empty variable.
• Fixed a broken File dependency when a plugin was set to absent and ::elasticsearch set to present.
• Fixed issue when using the proxy parameter on plugins in Elasticsearch 2.x.

Changes

• The api_ca_file and api_ca_path parameters have been added to support custom CA bundles for API access.
• Numerics in elasticsearch.yml will always be properly unquoted.
• puppetlabs/java is now listed as a dependency in metadata.json to avoid unexpected installation problems.

Testing changes

0.13.2 (August 29, 2016)

Summary

Primarily a bugfix release to resolve HTTPS use in elasticsearch::template resources, 5.x plugin operations, and plugin file permission enforcement.

Features

• Plugin installation for the 5.x series of Elasticsearch is now properly supported.

Bugfixes

• Recursively enforce correct plugin directory mode to avoid Elasticsearch startup permissions errors.
• Fixed an edge case where dependency cycles could arise when managing absent resources.
• Elasticsearch templates now properly use HTTPS when instructed to do so.

Changes

• Updated the elasticsearch_template type to return more helpful error output.
• Updated the es_instance_conn_validator type to silence deprecation warnings in Puppet >= 4.

Testing changes

0.13.1 (August 8, 2016)

Summary

Lingering bugfixes from elasticsearch::template changes. More robust systemd mask handling. Updated some upstream module parameters for deprecation warnings. Support for the Shield system_key file.

Features

• Added system_key parameter to the elasticsearch class and elasticsearch::instance type for placing Shield system keys.

Bugfixes

• Fixed systemd elasticsearch.service unit masking to use systemctl rather than raw symlinking to avoid puppet file backup errors.
• Fixed a couple of cases that broke compatability with older versions of puppet (elasticsearch_template types on puppet versions prior to 3.6 and yumrepo parameters on puppet versions prior to 3.5.1)
• Fixed issues that caused templates to be incorrectly detected as out-of-sync and thus always changed on each puppet run.
• Resources are now explicitly ordered to ensure behavior such as plugins being installed before instance start, users managed before templates changed, etc.

Changes

• Updated repository gpg fingerprint key to long form to silence module warnings.

Testing changes

0.13.0 (August 1, 2016)

Summary

Rewritten elasticsearch::template using native type and provider. Fixed and added additional proxy parameters to elasticsearch::plugin instances. Exposed repo priority parameters for apt and yum repos.

Features

• In addition to better consistency, the elasticsearch::template type now also accepts various api_* parameters to control how access to the Elasticsearch API is configured (there are top-level parameters that are inherited and can be overwritten in elasticsearch::api_*).
• The elasticsearch::config parameter now supports deep hiera merging.
• Added the elasticsearch::repo_priority parameter to support apt and yum repository priority configuration.
• Added proxy_username and proxy_password parameters to elasticsearch::plugin.

Bugfixes

• Content of templates should now properly trigger new API PUT requests when the index template stored in Elasticsearch differs from the template defined in puppet.
• Installing plugins with proxy parameters now works correctly due to changed Java property flags.
• The elasticsearch::plugin::module_dir parameter has been re-implemented to aid in working around plugins with non-standard plugin directories.

Changes

• The file parameter on the elasticsearch::template defined type has been deprecated to be consistent with usage of the source parameter for other types.

Testing changes

0.12.0 (July 20, 2016)

IMPORTANT! A bug was fixed that mistakenly added /var/lib to the list of DATA_DIR paths on Debian-based systems. This release removes that environment variable, which could potentially change path.data directories for instances of Elasticsearch. Take proper precautions when upgrading to avoid unexpected downtime or data loss (test module upgrades, et cetera).

Summary

Rewritten yaml generator, code cleanup, and various bugfixes. Configuration file yaml no longer nested. Service no longer restarts by default, and exposes more granular restart options.

Features

• The additional parameters restart_config_change, restart_package_change, and restart_plugin_change have been added for more granular control over service restarts.

Bugfixes

• Special yaml cases such as arrays of hashes and strings like "::" are properly supported.
• Previous Debian SysV init scripts mistakenly set the DATA_DIR environment variable to a non-default value.
• Some plugins failed installation due to capitalization munging, the elasticsearch_plugin provider no longer forces downcasing.

Changes

• The install_options parameter on the elasticsearch::plugin type has been removed. This was an undocumented parameter that often caused problems for users.
• The elasticsearch.service systemd unit is no longer removed but masked by default, effectively hiding it from systemd but retaining the upstream vendor unit on disk for package management consistency.
• restart_on_change now defaults to false to reduce unexpected cluster downtime (can be set to true if desired).
• Package pinning is now contained within a separate class, so users can opt to manage package repositories manually and still use this module's pinning feature.
• All configuration hashes are now flattened into dot-notated yaml in the elasticsearch configuration file. This should be fairly transparent in terms of behavior, though the config file formatting will change.

Testing changes

• The acceptance test suite has been dramatically slimmed to cut down on testing time and reduce false positives.

0.11.0 ( May 23, 2016 )

Summary

Shield support, SLES support, and overhauled testing setup.

Features

• Support for shield
  • TLS Certificate management
  • Users (role and password management for file-based realms)
  • Roles (file-based with mapping support)
• Support (repository proxies)[https://github.com/elastic/puppet-elasticsearch/pull/615]
• Support for (SSL auth on API calls)[https://github.com/elastic/puppet-elasticsearch/pull/577]

Bugfixes

• (Fix Facter calls)[https://github.com/elastic/puppet-elasticsearch/pull/590] in custom providers

Changes

Testing changes

• Overhaul testing methodology, see CONTRIBUTING for updates
• Add SLES 12, Oracle 6, and PE 2016.1.1 to testing matrix
• Enforce strict variable checking

Known bugs

• This is the first release with Shield support, some untested edge cases may exist

##0.10.3 ( Feb 08, 2016 )

###Summary Adding support for OpenBSD and minor fixes

####Features

• Add required changes to work with ES 2.2.x plugins
• Support for custom log directory
• Support for OpenBSD

####Bugfixes

• Add correct relation to file resource and plugin installation
• Notify service when upgrading the package

####Changes

• Remove plugin dir when upgrading Elasticsearch

####Testing changes

####Known bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.10.2 ( Jan 19, 2016 )

###Summary Bugfix release and adding Gentoo support

####Features

• Added Gentoo support

####Bugfixes

• Create init script when set to unmanaged
• init_template variable was not passed on correctly to other classes / defines
• Fix issue with plugin type that caused run to stall
• Export ES_GC_LOG_FILE in init scripts

####Changes

• Improve documentation about init_defaults
• Update common files
• Removed recurse option on data directory management
• Add retry functionality to plugin type

####Testing changes

####Known bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.10.1 ( Dec 17, 2015 )

###Summary Bugfix release for proxy functionality in plugin installation

####Features

####Bugfixes

• Proxy settings were not passed on correctly

####Changes

• Cleanup .pmtignore to exclude more files

####Testing changes

####Known bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.10.0 ( Dec 14, 2015 )

###Summary Module now works with ES 2.x completely

####Features

• Work with ES 2.x new plugin system and remain to work with 1.x
• Implemented datacat module from Richard Clamp so other modules can hook into it for adding configuration options
• Fixed init and systemd files to work with 1.x and 2.x
• Made the module work with newer pl-apt module versions
• Export es_include so it is passed on to ES
• Ability to supply long gpg key for apt repo

####Bugfixes

• Documentation and typographical fixes
• Do not force puppet:/// schema resource
• Use package resource defaults rather than setting provider and source

####Changes

####Testing changes

• Improve unit testing and shorten the runtime

####Known bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.9.9 ( Sep 01, 2015 )

###Summary Bugfix release and extra features

####Features

• Work with ES 2.x
• Add Java 8 detection in debian init script
• Improve offline plugin installation

####Bugfixes

• Fix a bug with new ruby versions but older puppet versions causing type error
• Fix config tempate to use correct ruby scoping
• Fix regex retrieving proxy port while downloading plugin
• Fix systemd template for better variable handling
• Template define was using wrong pathing for removal

####Changes

####Testing changes

####Known bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.9.8 ( Jul 07, 2015 )

###Summary

####Features

• Work with ES 2.x

####Bugfixes

• Fix plugin to maintain backwards compatibility

####Changes

####Testing changes

• ensure testing works with Puppet 4.x ( Rspec and Acceptance )

####Known bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.9.7 ( Jun 24, 2015 )

###Summary This releases adds several important features and fixes an important plugin installation issue with ES 1.6 and higher.

####Features

• Automate plugin dir extraction
• use init service provider for Amazon Linux
• Add Puppetlabs/apt and ceritsc/yum as required modules
• Added Timeout to fetching facts in case ES does not respond
• Add proxy settings for package download

####Bugfixes

• Fixed systemd template to fix issue with LimitMEMLOCK setting
• Improve package version handling when specifying a version
• Add tmpfiles.d file to manage sub dir in /var/run path
• Fix plugin installations for ES 1.6 and higher

####Changes

• Removed Modulefile, only maintaining metadata.json file

####Testing changes

• Added unit testing for package pinning feature
• Added integration testing with Elasticsearch to find issues earlier
• Fix OpenSuse 13 testing

####Known bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.9.6 ( May 28, 2015 )

###Summary Bugfix release 0.9.6

####Features

• Implemented package version pinning to avoid accidental upgrading
• Added support for Debian 8
• Added support for upgrading plugins
• Managing LimitNOFILE and LimitMEMLOCK settings in systemd

####Bugfixes

####Changes

• Dropped official support for PE 3.1.x and 3.2.x

####Testing changes

• Several testing changes implemented to increase coverage

####Known bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.9.5( Apr 16, 2015 )

###Summary Bugfix release 0.9.5

We reverted the change that implemented the full 40 character for the apt repo key. This caused issues with some older versions of the puppetlabs-apt module

####Features

####Bugfixes

• Revert using the full 40 character for the apt repo key.

####Changes

####Testing changes

####Known bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.9.4( Apr 14, 2015 )

###Summary Bugfix release 0.9.4

####Features

• Add the ability to create and populate scripts

####Bugfixes

• add support for init_defaults_file to elasticsearch::instance
• Update apt key to full 40characters

####Changes

• Fix readme regarding module_dir with plugins

####Testing changes

• Adding staged removal test
• Convert git urls to https
• Add centos7 node config

####Known bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.9.3( Mar 24, 2015 )

###Summary Bugfix release 0.9.3

####Features

####Bugfixes

• Not setting repo_version did not give the correct error
• Systemd file did not contain User/Group values

####Changes

• Brand rename from Elasticsearch to Elastic

####Testing changes

• Moved from multiple Gemfiles to single Gemfile

####Known bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.9.2( Mar 06, 2015 )

###Summary Bugfix release 0.9.2

####Features

• Introducing es_instance_conn_validator resource to verify instance availability

####Bugfixes

• Fix missing data path when using the path config setting but not setting the data path

####Changes None

####Testing changes None

####Known bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.9.1 ( Feb 23, 2015 )

###Summary This is the first bug fix release for 0.9 version. A bug was reported with the recursive file management.

####Features None

####Bugfixes

• Fix recursive file management
• Set undefined variables to work with strict_variables

####Changes None

####Testing changes None

####Known bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.9.0 ( Feb 02, 2015 )

###Summary This release is the first one towards 1.0 release. Our planning is to provide LTS releases with the puppet module

####Features

• Support for using hiera to define instances and plugins.
• Support for OpenSuSE 13.x
• Custom facts about the installed Elasticsearch instance(s)
• Proxy host/port support for the plugin installation
• Ability to supply a custom logging.yml template

####Bugfixes

• Ensure file owners are correct accross all related files
• Fix of possible service name conflict
• Empty main config would fail with instances
• Removal of standard files from packages we dont use
• Ensuring correct sequence of plugin and template defines
• Added ES_CLASSPATH export to init scripts

####Changes

• Java installation to use puppetlabs-java module
• Added Support and testing for Puppet 3.7 and PE 3.7
• Improve metadata.json based on scoring from Forge

####Testing changes

• Added testing against Puppet 3.7 and PE 3.7
• Using rspec3
• Using rspec-puppet-facts gem simplifies rspec testing

####Known Bugs

• Possible package conflicts when using ruby/python defines with main package name

##0.4.0 ( Jun 18, 2014 ) - Backwards compatible breaking release

###Summary This release introduces instances to facilitate the option to have more then a single instance running on the host system.

####Features

• Rewrite module to incorperate multi instance support
• New readme layout

####Bugfixes

• None

####Changes

• Adding ec2-linux osfamily for repo management
• Retry behaviour for plugin installation

####Testing changes

• Adding Puppet 3.6.x testing
• Ubuntu 14.04 testing
• Using new docker images
• Pin rspec to 2.14.x

####Known Bugs

• No known bugs

##0.3.2 ( May 15, 2014 )

• Add support for SLC/Scientific Linux CERN ( PR #121 )
• Add support for custom package names ( PR #122 )
• Fix python and ruby client defines to avoid name clashes.
• Add ability to use stage instead of anchor for repo class
• Minor fixes to system tests

##0.3.1 ( April 22, 2014 )

• Ensure we create the plugin directory before installing plugins
• Added Puppet 3.5.x to rspec and system tests

##0.3.0 ( April 2, 2014 )

• Fix minor issue with yumrepo in repo class ( PR #92 )
• Implement OpenSuse support
• Implement Junit reporting for tests
• Adding more system tests and convert to Docker images
• Use Augeas for managing the defaults file
• Add retry to package download exec
• Add management to manage the logging.yml file
• Improve inline documentation
• Improve support for Debian 6
• Improve augeas for values with spaces
• Run plugin install as ES user ( PR #108 )
• Fix rights for the plugin directory
• Pin Rake for Ruby 1.8.7
• Adding new metadata for Forge.
• Increase time for retry to insert the template

##0.2.4 ( Feb 21, 2014 )

• Set puppetlabs-stdlib dependency version from 3.0.0 to 3.2.0 to be inline with other modules
• Let puppet run fail when template insert fails
• Documentation improvements ( PR #77, #78, #83 )
• Added beaker system tests
• Fixed template define after failing system tests
• Some fixes so variables are more inline with intended structure

##0.2.3 ( Feb 06, 2014 )

• Add repository management feature
• Improve testing coverage and implement basic resource coverage reporting
• Add puppet 3.4.x testing
• Fix dependency in template define ( PR #72 )
• For apt repo change from key server to key file

##0.2.2 ( Jan 23, 2014 )

• Ensure exec names are unique. This caused issues when using our logstash module
• Add spec tests for plugin define

##0.2.1 ( Jan 22, 2014 )

• Simplify the management of the defaults file ( PR #64 )
• Doc improvements for the plugin define ( PR #66 )
• Allow creation of data directory ( PR #68 )
• Fail early when package version and package_url are defined

##0.2.0 ( Nov 19, 2013 )

• Large rewrite of the entire module described below
• Make the core more dynamic for different service providers and multi instance capable
• Add better testing and devided into different files
• Fix template function. Replace of template is now only done when the file is changed
• Add different ways to install the package except from the repository ( puppet/http/https/ftp/file )
• Update java class to install openjdk 1.7
• Add tests for python function
• Update config file template to fix scoping issue ( from PR #57 )
• Add validation of templates
• Small changes for preperation for system tests
• Update readme for new functionality
• Added more test scenario's
• Added puppet parser validate task for added checking
• Ensure we don't add stuff when removing the module
• Update python client define
• Add ruby client define
• Add tests for ruby clients and update python client tests

##0.1.3 ( Sep 06, 2013 )

• Exec path settings has been updated to fix warnings ( PR #37, #47 )
• Adding define to install python bindings ( PR #43 )
• Scope deprecation fixes ( PR #41 )
• feature to install plugins ( PR #40 )

##0.1.2 ( Jun 21, 2013 )

• Update rake file to ignore the param inherit
• Added missing documentation to the template define
• Fix for template define to allow multiple templates ( PR #36 by Bruce Morrison )

##0.1.1 ( Jun 14, 2013 )

• Add Oracle Linux to the OS list ( PR #25 by Stas Alekseev )
• Respect the restart_on_change on the defaults ( PR #29 by Simon Effenberg )
• Make sure the config can be empty as advertised in the readme
• Remove dependency cycle when the defaults file is updated ( PR #31 by Bruce Morrison )
• Enable retry on the template insert in case ES isn't started yet ( PR #32 by Bruce Morrison )
• Update templates to avoid deprecation notice with Puppet 3.2.x
• Update template define to avoid auto insert issue with ES
• Update spec tests to reflect changes to template define

##0.1.0 ( May 09, 2013 )

• Populate .gitignore ( PR #19 by Igor Galić )
• Add ability to install initfile ( PR #20 by Justin Lambert )
• Add ability to manage default file service parameters ( PR #21 by Mathieu Bornoz )
• Providing complete containment of the module ( PR #24 by Brian Lalor )
• Add ability to specify package version ( PR #25 by Justin Lambert )
• Adding license file

##0.0.7 ( Mar 23, 2013 )

• Ensure config directory is created and managed ( PR #13 by Martin Seener )
• Dont backup package if it changes
• Create explicit dependency on template directory ( PR #16 by Igor Galić )
• Make the config directory variable ( PR #17 by Igor Galić and PR #18 by Vincent Janelle )
• Fixing template define

##0.0.6 ( Mar 05, 2013 )

• Fixing issue with configuration not printing out arrays
• New feature to write the config hash shorter
• Updated readme to reflect the new feature
• Adding spec tests for config file generation

##0.0.5 ( Mar 03, 2013 )

• Option to disable restart on config file change ( PR #10 by Chris Boulton )

##0.0.4 ( Mar 02, 2013 )

• Fixed a major issue with the config template ( Issue #9 )

##0.0.3 ( Mar 02, 2013 )

• Adding spec tests
• Fixed init issue on Ubuntu ( Issue #6 by Marcus Furlong )
• Fixed config template problem ( Issue #8 by surfchris )
• New feature to manage templates

##0.0.2 ( Feb 16, 2013 )

• Feature to supply a package instead of being dependent on the repository
• Feature to install java in case one doesn't manage it externally
• Adding RedHat and Amazon as Operating systems
• fixed a typo - its a shard not a shared :) ( PR #5 by Martin Seener )

##0.0.1 ( Jan 13, 2013 )

• Initial release of the module