Forge Home

filebeat

This module installs, configures and manages the Filebeat service.

17,019 downloads

15,437 latest version

5.0 quality score

We run a couple of automated
scans to help you access a
module's quality. Each module is
given a score based on how well
the author has formatted their
code and documentation and
modules are also checked for
malware using VirusTotal.

Please note, the information below
is for guidance only and neither of
these methods should be considered
an endorsement by Puppet.

Version information

  • 1.0.2 (latest)
  • 1.0.1
  • 1.0.0
released Jan 28th 2019
This version is compatible with:
  • Puppet Enterprise 2018.1.x, 2017.3.x, 2017.2.x, 2017.1.x, 2016.5.x, 2016.4.x
  • Puppet >= 4.7.0 < 6.0.0
  • , , , , , ,

Start using this module

  • r10k or Code Manager
  • Bolt
  • Manual installation
  • Direct download

Add this module to your Puppetfile:

mod 'slauger-filebeat', '1.0.2'
Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add slauger-filebeat
Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install slauger-filebeat --version 1.0.2

Direct download is not typically how you would use a Puppet module to manage your infrastructure, but you may want to download the module in order to inspect the code.

Download

Documentation

slauger/filebeat — version 1.0.2 Jan 28th 2019

puppet-filebeat

Table of Contents

  1. Module Description - What the module does and why it is useful
  2. Setup - The basics of getting started with filebeat
  3. Usage - Configuration options and additional functionality
  4. Reference - An under-the-hood peek at what the module is doing and how
  5. Limitations - OS compatibility, etc.
  6. Development - Guide for contributing to the module

Module description

The filebeat module installs, configures and manages the Elastic filebeat service for shipping logs to Logstash, Kafka or Elasticsearch.

Setup

Filebeat packages

This modules currently does not manage the package repository for the elastic. The main reason for this is that the module is primarily written for enterprise environments, without an direct access to the public internet. Furthermore, as you also might want to install Logstash, Elasticsearch and Kibana in your in environment (and there is only a "all-in-one" package repository for elastic), there would be a high chance of a duplicate resource definition of the package repository.

If you don't have a central repository management in your environment (like Pulp or RedHat Satellite) prepare the package repository on your nodes with a different module (e.g. elastic_stack), or use the module parameters to switch to an "url based" installation method.

filebeat::package_provider: 'rpm'
filebeat::package_source: 'https://artifacts.elastic.co/packages/6.x/yum/6.4.0/filebeat-6.4.0-x86_64.rpm'

Configure a output

Filebeat requires at least one configured output. See below for example configurations.

Usage

All parameters for the filebeat module are contained within the main filebeat class, so for any function of the module, set the options you want. See the common usages below for examples.

Install and enable filebeat

include filebeat

Define a filebeat input

Each hash entry represents a own configuration file inside of /etc/filebeat/inputs.d.

filebeat::inputs:
  syslog:
    - type: log
      enabled: true
      paths:
        - /var/log/messages
        - /var/log/syslog

Optional additional fields can be added.

filebeat::inputs:
  syslog:
    - type: log
      enabled: true
      paths:
        - /var/log/messages
        - /var/log/syslog
      fields:
        type: syslog

Configure a filebeat module

Each hash entry represents a own configuration file inside of /etc/filebeat/modules.d.

filebeat::modules:
  'system':
    - module: system
      syslog:
        enabled: true
      auth:
        enabled: true

Configure Logstash output

filebeat::config:
  output:
    logstash:
      hosts:
        - 'logstash1.example.com:5044'
        - 'logstash2.example.com:5044'

Configure Apache Kafka output

filebeat::config:
  output:
    kafka:
      hosts:
        - 'kafka1.example.com:9092'
        - 'kafka2.example.com:9092'
      topic: 'logs'

Configure Apache Kafka with output to different topics

filebeat::inputs:
  syslog:
    - type: log
      enabled: true
      paths:
        - /var/log/messages
        - /var/log/syslog
      fields:
        kafka_topic: syslog
  apache:
    - type: log
      enabled: true
      paths:
        - /var/log/httpd/access_log
        - /var/log/apache2/access_log
      fields:
        kafka_topic: apache

filebeat::config:
  output:
    kafka:
      hosts:
        - 'kafka1.example.com:9092'
        - 'kafka2.example.com:9092'
      topic: "%{literal('%')}{[fields.kafka_topic]}"

Configure Elasticsearch output

filebeat::config:
  output:
    elasticsearch:
      hosts: ["localhost:9200"]
      protocol: "https"
      username: "elastic"
      password: "changeme"

Full example configuration

A complete example configuration using the Logstash output with authentication via the Puppet SSL certificates TLS hardening.

filebeat::env:
  'APPLICATION': 'puppetserver'

filebeat::config:
  name: '%{facts.fqdn}'
  tags:
    - '%{facts.osfamily}'
  fields:
    environment: '%{environment}'
    application: '${APPLICATION:none}'
  logging.level: 'error'
  logging.selectors: ["*"]
  output:
    logstash:
      hosts: ["logstash.example.com:5044"]
      ssl.certificate_authorities: ["/etc/puppetlabs/puppet/ssl/certs/ca.pem"]
      ssl.certificate: "/etc/puppetlabs/puppet/ssl/certs/%{trusted.certname}.pem"
      ssl.key: "/etc/puppetlabs/puppet/ssl/private_keys/%{trusted.certname}.pem"
      ssl.supported_protocols: [TLSv1.2]
      ssl.cipher_suites: [ECDHE-ECDSA-AES-256-GCM-SHA384, ECDHE-RSA-AES-256-GCM-SHA384, ECDHE-ECDSA-AES-128-GCM-SHA256, ECDHE-RSA-AES-128-GCM-SHA256]
      ssl.curve_types: [P-256, P-384, P-521]
      ssl.renegotiation: never

filebeat::modules:
  system:
    - module: system
      syslog:
        enabled: true
      auth:
        enabled: true

filebeat::inputs:
  apache:
    - type: log
      enabled: true
      paths:
        - /var/log/httpd/*_access.log
        - /var/log/httpd/*_access_ssl.log
      fields:
        type: apache_access

Reference

See REFERENCE.md

Limitations

For an extensive list of supported operating systems, see metadata.json

Development

Puppet modules on the Puppet Forge are open projects, and community contributions are essential for keeping them great. Feature requests and pull requests are appreciated.

Contributors

To see who's already involved, see the list of contributors.