filebeat

This module installs, configures and manages the Filebeat service.

Module Stats

18,151 downloads

16,533 latest version

5.0 quality score

Support the Puppet Community by contributing to this module

You are welcome to contribute to this module by suggesting new features, currency updates, or fixes. Every contribution is valuable to help ensure that the module remains compatible with the latest Puppet versions and continues to meet community needs. Complete the following steps:

Review the module’s contribution guidelines and any licenses. Ensure that your planned contribution aligns with the author’s standards and any legal requirements.
Fork the repository on GitHub, make changes on a branch of your fork, and submit a pull request. The pull request must clearly document your proposed change.

For questions about updating the module, contact the module’s author.

Version information

released Jan 28th 2019

This version is compatible with:

Puppet Enterprise 2018.1.x, 2017.3.x, 2017.2.x, 2017.1.x, 2016.5.x, 2016.4.x
Puppet >= 4.7.0 < 6.0.0
, , , , , ,

Start using this module

Installation method

Add this module to your Puppetfile:

mod 'slauger-filebeat', '1.0.2'

Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add slauger-filebeat

Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install slauger-filebeat --version 1.0.2

Documentation

slauger/filebeat — version 1.0.2 Jan 28th 2019

puppet-filebeat

Module Description - What the module does and why it is useful
Setup - The basics of getting started with filebeat
Usage - Configuration options and additional functionality
Reference - An under-the-hood peek at what the module is doing and how
Limitations - OS compatibility, etc.
Development - Guide for contributing to the module

Module description

The filebeat module installs, configures and manages the Elastic filebeat service for shipping logs to Logstash, Kafka or Elasticsearch.

Setup

Filebeat packages

This modules currently does not manage the package repository for the elastic. The main reason for this is that the module is primarily written for enterprise environments, without an direct access to the public internet. Furthermore, as you also might want to install Logstash, Elasticsearch and Kibana in your in environment (and there is only a "all-in-one" package repository for elastic), there would be a high chance of a duplicate resource definition of the package repository.

If you don't have a central repository management in your environment (like Pulp or RedHat Satellite) prepare the package repository on your nodes with a different module (e.g. elastic_stack), or use the module parameters to switch to an "url based" installation method.

filebeat::package_provider: 'rpm'
filebeat::package_source: 'https://artifacts.elastic.co/packages/6.x/yum/6.4.0/filebeat-6.4.0-x86_64.rpm'

Configure a output

Filebeat requires at least one configured output. See below for example configurations.

Usage

All parameters for the filebeat module are contained within the main filebeat class, so for any function of the module, set the options you want. See the common usages below for examples.

Install and enable filebeat

include filebeat

Define a filebeat input

Each hash entry represents a own configuration file inside of /etc/filebeat/inputs.d.

filebeat::inputs:
  syslog:
    - type: log
      enabled: true
      paths:
        - /var/log/messages
        - /var/log/syslog

Optional additional fields can be added.

filebeat::inputs:
  syslog:
    - type: log
      enabled: true
      paths:
        - /var/log/messages
        - /var/log/syslog
      fields:
        type: syslog

Configure a filebeat module

Each hash entry represents a own configuration file inside of /etc/filebeat/modules.d.

filebeat::modules:
  'system':
    - module: system
      syslog:
        enabled: true
      auth:
        enabled: true

Configure Logstash output

filebeat::config:
  output:
    logstash:
      hosts:
        - 'logstash1.example.com:5044'
        - 'logstash2.example.com:5044'

Configure Apache Kafka output

filebeat::config:
  output:
    kafka:
      hosts:
        - 'kafka1.example.com:9092'
        - 'kafka2.example.com:9092'
      topic: 'logs'

Configure Apache Kafka with output to different topics

filebeat::inputs:
  syslog:
    - type: log
      enabled: true
      paths:
        - /var/log/messages
        - /var/log/syslog
      fields:
        kafka_topic: syslog
  apache:
    - type: log
      enabled: true
      paths:
        - /var/log/httpd/access_log
        - /var/log/apache2/access_log
      fields:
        kafka_topic: apache

filebeat::config:
  output:
    kafka:
      hosts:
        - 'kafka1.example.com:9092'
        - 'kafka2.example.com:9092'
      topic: "%{literal('%')}{[fields.kafka_topic]}"

Configure Elasticsearch output

filebeat::config:
  output:
    elasticsearch:
      hosts: ["localhost:9200"]
      protocol: "https"
      username: "elastic"
      password: "changeme"

Full example configuration

A complete example configuration using the Logstash output with authentication via the Puppet SSL certificates TLS hardening.

filebeat::env:
  'APPLICATION': 'puppetserver'

filebeat::config:
  name: '%{facts.fqdn}'
  tags:
    - '%{facts.osfamily}'
  fields:
    environment: '%{environment}'
    application: '${APPLICATION:none}'
  logging.level: 'error'
  logging.selectors: ["*"]
  output:
    logstash:
      hosts: ["logstash.example.com:5044"]
      ssl.certificate_authorities: ["/etc/puppetlabs/puppet/ssl/certs/ca.pem"]
      ssl.certificate: "/etc/puppetlabs/puppet/ssl/certs/%{trusted.certname}.pem"
      ssl.key: "/etc/puppetlabs/puppet/ssl/private_keys/%{trusted.certname}.pem"
      ssl.supported_protocols: [TLSv1.2]
      ssl.cipher_suites: [ECDHE-ECDSA-AES-256-GCM-SHA384, ECDHE-RSA-AES-256-GCM-SHA384, ECDHE-ECDSA-AES-128-GCM-SHA256, ECDHE-RSA-AES-128-GCM-SHA256]
      ssl.curve_types: [P-256, P-384, P-521]
      ssl.renegotiation: never

filebeat::modules:
  system:
    - module: system
      syslog:
        enabled: true
      auth:
        enabled: true

filebeat::inputs:
  apache:
    - type: log
      enabled: true
      paths:
        - /var/log/httpd/*_access.log
        - /var/log/httpd/*_access_ssl.log
      fields:
        type: apache_access

Reference

See REFERENCE.md

Limitations

For an extensive list of supported operating systems, see metadata.json

Development

Puppet modules on the Puppet Forge are open projects, and community contributions are essential for keeping them great. Feature requests and pull requests are appreciated.

Contributors

To see who's already involved, see the list of contributors.

Reference

Classes

Public Classes

filebeat: Main class, includes all other classes

Private Classes

filebeat::config: This class handles the configuration file.
filebeat::package: This class handles filebeat package.
filebeat::service: This class handles the filebeat service.

Defined types

filebeat::input: This defined type creates a new filebeat input.
filebeat::module: This defined type creates a new filebeat module.

Classes

filebeat

Parameters

The following parameters are available in the filebeat class.

`package_name`

Data type: String

Name of the filebeat package. Dfault value: 'filebeat'.

`package_ensure`

Data type: String

State of package (e.g. present oder latest). Default value: 'present'.

`package_provider`

Data type: Optional[String]

Optional package provider for the package. Default value: Undef.

`package_source`

Data type: Optional[String]

Optional package source for the package. Default value: Undef.

`service_name`

Data type: String

Name of the filebeat service. Default value: 'filebeat'.

`service_ensure`

Data type: Optional[String]

Whether the filebeat service should be running. Default value: 'running'.

`service_enable`

Data type: Optional[Boolean]

Whether the filebeat service should be enabled. Default value: true.

`environment_file`

Data type: String

Path to the environment file for systemd. Default value: '/etc/sysconfig/filebeat'.

`inputs`

Data type: Optional[Hash]

An hash with enabled inputs for filebeat. Default value: {}.

`modules`

Data type: Optional[Hash]

An hash with enabled modules for filebeat. Default value: {}.

`config`

Data type: Optional[Hash]

An hash with the main and output configuration. Default value: {}.

`env`

Data type: Optional[Hash]

An array with environment variables, wich will be written to the environment_file. Default value: [].

Defined types

filebeat::input

This defined type creates a new filebeat input.

Parameters

The following parameters are available in the filebeat::input defined type.

`config`

Data type: Array

An array with input configurations.

filebeat::module

This defined type creates a new filebeat module.

Parameters

The following parameters are available in the filebeat::module defined type.

`config`

Data type: Array

An array with module configurations.

Modules

Discover

Contribute

Puppet

Premium features

Downloads

Resources

About Forge

Getting Started

filebeat

Support the Puppet Community by contributing to this module

Version information

This version is compatible with:

Start using this module

Documentation

puppet-filebeat

Table of Contents

Module description

Setup

Filebeat packages

Configure a output

Usage

Install and enable filebeat

Define a filebeat input

Configure a filebeat module

Configure Logstash output

Configure Apache Kafka output

Configure Apache Kafka with output to different topics

Configure Elasticsearch output

Full example configuration

Reference

Limitations

Development

Contributors

Reference

Table of Contents

Classes

filebeat

Parameters

package_name

package_ensure

package_provider

package_source

service_name

service_ensure

service_enable

environment_file

inputs

modules

config

env

Defined types

filebeat::input

Parameters

config

filebeat::module

Parameters

config

Change log

v1.0.1 (2018-10-02)

v1.0.0 (2018-09-14)

Dependencies

`package_name`

`package_ensure`

`package_provider`

`package_source`

`service_name`

`service_ensure`

`service_enable`

`environment_file`

`inputs`

`modules`

`config`

`env`

`config`

`config`