Forge Home

Modules

Puppet

Resources

PuppetEcosystemPuppet CommunityUpdates

logstash

Module for managing and configuring Logstash
Project URLRSS FeedReport issues

2,784,036 downloads

674,318 latest version

4.2 quality score

We run a couple of automated
scans to help you access a
module's quality. Each module is
given a score based on how well
the author has formatted their
code and documentation and
modules are also checked for
malware using VirusTotal.

Please note, the information below
is for guidance only and neither of
these methods should be considered
an endorsement by Puppet.

Version information

  • 6.1.5 (latest)
  • 6.1.4
  • 6.1.3
  • 6.1.2
  • 6.1.1
  • 6.1.0
  • 6.0.2
  • 6.0.1
  • 6.0.0
  • 5.3.0
  • 5.2.1
  • 5.2.0
  • 5.1.0
  • 5.0.4
  • 5.0.3
  • 5.0.2
  • 5.0.1
  • 5.0.0
released Dec 18th 2016
This version is compatible with:
  • Puppet Enterprise >= 3.8.6 <5.0.0
  • Puppet >=3.8.6 <5.0.0
  • , , , , , ,
This module has been deprecated by its author since Jan 5th 2023.

The author has suggested puppet-logstash as its replacement.

Start using this module

Documentation

elastic/logstash — version 5.0.2 Dec 18th 2016

elastic/logstash

A Puppet module for managing and configuring Logstash.

Build Status

Logstash Versions

This module, "elastic/logstash" supports only Logstash 5.x. For earlier Logstash versions, support is provided by the legacy module, "elasticsearch/logstash".

Requirements

  • Puppet 3.8.6 or better.
  • The stdlib Puppet library.

Optional:

  • The apt (>= 2.0.0) Puppet library when using repo management on Debian/Ubuntu.
  • The zypprepo Puppet library when using repo management on SLES/SuSE

Quick Start

This minimum viable configuration ensures that the service is running will be started at boot time.

include logstash

# You must provide a valid pipeline configuration for the service to start.
logstash::configfile { 'my_ls_config':
  content => template('path/to/config.file'),
}

Package and service options

Choosing a Logstash minor version

class { 'logstash':
    version => '5.0.2',
}

Manual repository management

You may want to manage repositories manually. You can disable automatic repository management like this:

class { 'logstash':
  manage_repo  => false,
}

Using an explicit package source

Rather than use your distribution's repository system, you can specify an explicit package to fetch and install.

From an HTTP/HTTPS/FTP URL

class { 'logstash':
  package_url => 'https://artifacts.elastic.co/downloads/logstash/logstash-5.1.1.rpm',
}

From a 'puppet://' URL

class { 'logstash':
  package_url => 'puppet:///modules/my_module/logstash-5.1.1.rpm',
}

From a local file on the agent

class { 'logstash':
  package_url => 'file:///tmp/logstash-5.1.1.rpm',
}

Allow automatic point-release upgrades

class { 'logstash':
  auto_upgrade  => true,
}

Do not run as a service

class { 'logstash':
  status => 'disabled',
}

Disable automatic restarts

Under normal circumstances, changing a configuration will trigger a restart of the service. This behaviour can be disabled:

class { 'logstash':
  restart_on_change => false,
}

Disable and remove Logstash

class { 'logstash':
  ensure => 'absent',
}

Logstash config files

Settings

Logstash uses several files to define settings for the service and associated Java runtime. The settings files can be configured with class parameters.

logstash.yml

class { 'logstash':
  settings => {
    'pipeline.batch.size'  => 25,
    'pipeline.batch.delay' => 5,
  },
}

jvm.options

class { 'logstash':
  jvm_options => [
    '-Xms1g',
    '-Xmx1g',
  ]
}

startup.options

class { 'logstash':
  startup_options => {
    'LS_NICE' => '10',
  }
}

Pipeline Configuration

Pipeline configuration files can be declared with the logstash::configfile type.

logstash::configfile { 'inputs':
  content => template('path/to/input.conf.erb'),
}

or

logstash::configfile { 'filters':
  source => 'puppet:///path/to/filter.conf',
}

For simple cases, it's possible to provide your Logstash config as an inline string:

logstash::configfile { 'basic_ls_config':
  content => 'input { heartbeat {} } output { null {} }',
}

If you want to use hiera to specify your configs, include the following create_resources call in your node manifest or in manifests/site.pp:

$logstash_configs = hiera('logstash_configs', {})
create_resources('logstash::configfile', $logstash_configs)

...and then include the following config within the corresponding hiera file:

"logstash_configs": {
  "config-name": {
    "template": "logstash/config.file.erb",
  }
}

Patterns

Many plugins (notably Grok) use patterns. While many are included in Logstash already, additional site-specific patterns can be managed as well; where possible, you are encouraged to contribute new patterns back to the community.

logstash::patternfile { 'extra_patterns':
  source => 'puppet:///path/to/extra_pattern',
}

By default the resulting filename of the pattern will match that of the source. This can be over-ridden:

logstash::patternfile { 'extra_patterns_firewall':
  source   => 'puppet:///path/to/extra_patterns_firewall_v1',
  filename => 'extra_patterns_firewall',
}

IMPORTANT NOTE: Using logstash::patternfile places new patterns in the correct directory, however, it does NOT cause the path to be included automatically for filters (example: grok filter). You will still need to include this path (by default, /etc/logstash/patterns/) explicitly in your configurations.

Example: If using 'grok' in one of your configurations, you must include the pattern path in each filter like this:

# Note: this example is Logstash configuration, not a Puppet resource.
# Logstash and Puppet look very similar!
grok {
  patterns_dir => "/etc/logstash/patterns/"
  ...
}

Plugin management

Installing by name (from RubyGems.org)

logstash::plugin { 'logstash-input-beats': }

Installing from a local Gem

logstash::plugin { 'logstash-input-custom':
  source => '/tmp/logstash-input-custom-0.1.0.gem',
}

Installing from a 'puppet://' URL

logstash::plugin { 'logstash-filter-custom':
  source => 'puppet:///modules/my_ls_module/logstash-filter-custom-0.1.0.gem',
}

Support

Need help? Join us in #logstash on Freenode IRC or on the https://discuss.elastic.co/c/logstash discussion forum.