packetbeat

Table of Contents

1. Description
2. Setup - The basics of getting started with packetbeat
   • What packetbeat affects
   • Beginning with packetbeat
3. Usage - Configuration options and additional functionality
   • Processors
4. Reference - An under-the-hood peek at what the module is doing and how
   • Public Classes
   • Private Classes
5. Limitations - OS compatibility, etc.
   • Major Versions
6. Development - Guide for contributing to the module
   • Testing

Description

The packetbeat module installs the packetbeat network packet analyzer maintained by elastic.

Setup

What packetbeat affects

By default packetbeat adds a software repository to your system and installs packetbeat along with the required configurations.

Beginning with packetbeat

packetbeat requires the protocols and outputs parameters to be declared, without which the service does nothing.

class{'packetbeat':
  protocols => {
    'icmp' => {
      'enabled' => true,
    },
  },
  outputs   => {
    'elasticsearch' => {
      'hosts' => ['localhost:9200'],
    },
  }
}

Usage

As of this writing all the default values follow the upstream values. This module saves all configuration options in a to_yaml() fashion, therefore multiple instances of the same protocol are not possible.

To ship HTTP traffic to elasticsearch

class{'packetbeat':
  protocols => {
    'http' => {
      'ports' => [80]
    }
  },
  outputs   => {
    'elasticsearch' => {
      'hosts' => ['localhost:9200']
    }
  }
}

To ship MySQL traffic through logstash

class{'packetbeat':
  protocols => {
    'mysql' => {
      'ports' => [3306]
    }
  },
  outputs   => {
    'logstash' => {
      'hosts' => ['localhost:5044'],
      'index' => 'packetbeat'
    }
  }
}

Network device configuation and logging can be configured the same way. Please review the documentation of the elastic website

Upgrading to 6.0

Version 0.2.0 of this module supports Packetbeat 6.0. Please review the Packetbeat Changelog for a full list of software changes and the Module Changelog for a list of module updates.

To upgrade existing installations:

class{'packetbeat':
  major_version  => '6',
  package_ensure => 'latest',
  ...
}

Processors

Libbeat 5.0 and later include a feature for filtering/enhancing exported data called processors. These may be added into the configuration by populating the processors parameter and may apply to all events or those that match certain conditions.

To drop events that have an http response code between 200 and 299

class{'packetbeat':
  processors => [
    {
      'drop_event' => {
        'when' => {
          'http.response.code.gte' => 200,
          'http.response.code.lt'  => 300
        }
      }
    }
  ],
  ...
}

To drop the mysql.num_fields field from the output

class{'packetbeat':
  processors => [
    {
      'drop_field' => {
        'fields' => 'mysql.num_fields'
      }
    }
  ]
}

For more information please review the documentation

Reference

• Public Classes
  • Class: packetbeat
• Private Classes
  • Class: packetbeat::config
  • Class: packetbeat::install
  • Class: packetbeat::repo
  • Class: packetbeat::service

Public Classes

Class: packetbeat

Installs and configures packetbeat.

Parameters within packetbeat

• outputs: [Hash] The required outputs section of the configuration.
• protocols: [Hash] The required protocols section of the configuration.
• ensure: [String] Valid values are 'present' and 'absent'. Determines weather to manage all required resources or remove them from the node. (default: 'present')
• beat_name: [String] The name of the beat shipper (default: hostname)
• bpf_filter: [String] Overwrite packetbeat's automatically generated BPF with this value. This setting is only available if type is configured for 'af_packet'. NOTE: It is the responsibility of the user to ensure this is in-sync with the protocols.
• buffer_size_mb: [Integer] The maximum size of the shared memory buffer to use between the kernel and user-space. This setting is only available if type is configured for 'af_packet'.
• config_file_mode: [String] The octal permissions to set on configuration files. (default: '0644')
• device: [String] The name of the interface from which to capture traffic. (default: 'any')
• disable_config_test: [Boolean] If true, disable configuration file testing. It is generally recommended to leave this parameter at this default value. (default: false)
• fields: [Hash] Optional fields to add any additional information to the output. (default: undef)
• fields_under_root: [Boolean] By default custom fields are under a fields sub-dictionary. When set to true custom fields are added to the root-level document. (default: false)
• flow_enable: [Boolean] Enables or disables the bidirectional network flows. (default: true)
• flow_period: [String] Configures the reporting interval where all network flows are reported at the same time. This option takes a number followed by a time unit suffix, 's' representing seconds, 'm' representing minutes and so on. (default: '10s')
• flow_timeout: [String] Configures the lifetime of the flow. Like flow_period this option takes a number followed by a time-unit suffix. (default: '30s')
• logging: [Hash] Defines packetbeat's logging configuration, if not explicitly configured all logging output is forwarded to syslog on Linux nodes and file output on Windows. See the docs for all available options.
• major_version: [Enum] The major version of Packetbeat to install. Valid values are '5' and '6'. (default: '5')
• manage_repo: [Boolean] When false does not install the upstream repository to the node's package manager. (default: true)
• package_ensure: [String] The desired state of the Package resources. Only applicable if ensure is 'present'. (default: 'present')
• processors: [Array[Hash]] Add processors to the configuration to run on data before sending to the output. (default: undef)
• queue: [Hash] Configure the internal queue in packetbeat before being consumed by the output(s).
• queue_size: [Integer] The queue size for single events in the processing pipeline. This is only applicable if major_version is '5'. (default: 1000)
• service_ensure: [String] Determine the state of the packet beat service. Must be one of 'enabled', 'disabled', 'running', 'unmanaged'. (default: enabled)
• service_has_restart: [Boolean] When true the Service resource issues the 'restart' command instead of 'stop' and 'start'. (default: true)
• snaplen: [Integer] The maximum size of the packets to capture. Most environments can accept the default, on a physical interface the optimal value is the MTU size. (default: 65535)
• sniff_type: [String] Configure the sniffer type, packet beat only supports 'pcap', and 'af_packet' (Linux only, faster than 'pcap') (default: 'pcap')
• tags: [Array] Optional list of tags to help group different logical properties easily. (default: undef)
• with_vlans: [Boolean] If traffic contains VLAN tags all traffic is offset by four bits and packetbeat's internal BPF filter is ineffective. Only used if sniff_type is 'af_packet'. (default: undef)

Private Classes

Class: packetbeat::config

Manages packetbeats main configuration file.

Class: packetbeat::install

Installs the packetbeat package.

Class: packetbeat::repo

Installs the upstream Yum or Apt repository for the system package manager.

Class: packetbeat::service

Manages the packetbeat service.

Limitations

This module does not support loading kibana dashboards or elasticsearch templates, used when outputting to Elasticsearch.

Major Versions

This module was written for packetbeat versions 5.0 and greater. There is no supported for 1.x versions.

Development

Pull requests and bug reports are welcome. If you're sending a pull request, please consider writing tests if applicable.

Testing

Sandbox testing is done through the PDK utility provided by Puppet. To utilize PDK execute the following commands to validate and test the new code:

1. Validate syntax of metadata.json, all *.pp* and all *.rb files

pdk validate

2. Perform tests

pdk test unit

Changelog

All notable changes to this project will be documented in this file

[0.2.0]

• Adding support for Packetbeat 6.0 Removing unsupported sniffer type pf_ring from available sniff_type options Adding new parameter major_version to allow installation of 6.x packages from vendor ** Adding new optional parameter queue to configure internal queue settings
• Parameter queue_size is only applicable if major_version is '5'

0.1.1

This is a bug fix release with support for Puppet 5

Added

• Puppet 5 support
• Parameter disable_config_test, enabling users to opt-out of configuration file validation

Bug Fixes

• Packetbeat configuration file validation

Misc

• Replacing Pattern type parameters that allow specific values with Enum

0.1.0

Initial release, review module and documentation for configuration options

Added

Bug Fixes

Misc