syslog_ng

Table of Contents

• Overview
• Module Description
  • Configuration syntax
• Setup
  • Puppet Forge
  • Installing from source
  • What syslog_ng affects
  • Getting started with syslog_ng
• Usage
  • Facts
  • Classes and defined types
• Implementation details
• Limitations
• Development
  • Tests
  • Other information

Overview

This module lets you generate syslog-ng configuration using puppet. It supports all kinds of statements, such as sources, destinations, templates, and so on. After defining them, you can combine them into a log path. This module also takes care of installing syslog-ng, or reloading it after a configuration file change.

You can check the supported platforms in the Limitations section.

Module Description

This module integrates well with syslog-ng. It supports its configuration model so you can create new sources and destinations as Puppet resources. Under the hood they are just defined resource types.

The supported statements:

• options
• template
• rewrite
• parser
• filter (partial support)
• source
• destination
• log
• +1: config, which lets you insert existing configuration snippets.

Each type is under the syslog_ng:: namespace, so you can use them like this:

syslog_ng::source { 's_gsoc':
    params => {
        'type' => 'tcp',
        'options' => [
            {'ip' => "'127.0.0.1'"},
            {'port' => 1999}
        }]
    }
}

There is a shorter form:

syslog_ng::source { 's_gsoc':
    params => {
        'tcp' => [
            {'ip' => "'127.0.0.1'"},
            {'port' => 1999}
        }]
    }
}

Configuration syntax

Every statement has the same layout. They can accept a params parameter, which can be a hash or an array of hashes. Each hash should have a type and options key or you can use a shorter form.

The value of the type represents the type of the statement, in case of a source this can be file, tcp and so on.

The value of the options is an array of strings and hashes. You have to pay attention to quotation when using strings. If you want the inner quotation to be a single quote ( in the syslog-ng.conf), then the outer one must be a double, like "'this string'", which will be transformed into 'this string'.

Similarly, you can write '"this string"'' to get "this string" in the configuration.

By using this convention, the module will generate correct configuration files. If the option array is empty, you can use an empty string '' instead.

As I mentioned, there are strings and hashes in an option. Hashes must contain only one key. This key will identify the name of the parameter and its value must be an array of strings. If that would contain only one item, the value can be simply just a string.

You can find a lot of examples under the tests and spec directories. The default_config.pp under the tests directory contains the default configuration from the syslog-ng source, translated into Puppet types.

Setup

Puppet Forge

This module is published on Puppet Forge. It used to be under the ihrwein namespace, but the original author kindly accepted to hand it over.

Installing from source

You can install it following these steps:

0. Make sure you have the required dependencies

• ruby
• bundler

1. Clone the source code into a directory:

   $ git clone https://github.com/ccin2p3/puppet-syslog_ng.git
   

2. Make sure you are on master branch:

   $ git checkout master
   

3. Get dependencies

   $ bundle install
   

4. Build a package:

   $ bundle exec puppet module build
   

   This will create a tar.gz file under the pkg directory. Now you should be able to install the module:

   $ bundle exec puppet module install pkg/ccin2p3-syslog_ng-VERSION.tar.gz
   

What syslog_ng affects

• It setup a repository with recent syslog-ng releases (only on RedHat and Debian based operating systems and if $syslog_ng::manage_repo is set to true)
• It installs the syslog-ng or syslog-ng-core package
  • that creates the necessary directories on your system, including /etc/syslog-ng.
  • If another syslog daemon is installed, it will be removed by your package manager.
• purges the content of /etc/syslog-ng/syslog-ng.conf

Getting started with syslog_ng

If you are not familiar with syslog-ng, I suggest you to take a look at the Syslog-ng Admin Guide which contains all the necessary information to use this software.

You can also get help on the syslog-ng mailing list.

Usage

Just use the classes and defined types as you would use them, without Puppet.

Before the generated configuration would be applied, it is written to a temporary file. Next, the module checks the configuration syntax of this file, and if it is OK, it overwrites the real configuration file. So you do not have to worry about configuration errors.

Facts

The fact syslog_ng_version contains the installed version string e.g. 3.7.1

Classes and defined types

For information on the classes and types, see the REFERENCE.md.

Implementation details

There is a concat::fragment resource in every class or defined type which represents a statement. Because statements need to be defined before they are referenced in the configuration, I use an automatic ordering system. Each type has its own order value, which determines its position in the configuration file. The smaller an order value is, the more likely it will be at the beginning of the file. The interval of these values starts with 0 and are 'strings'. Here is a table, which contains the default order values:

The config generation is done by the generate_statement() function in most cases. It is just a wrapper around my statement.rb Ruby module, which does the hard work. The require part may seem quite ugly, but it works well.

Limitations

The module works well with the following Puppet versions:

• 2.7.9
• 2.7.13
• 2.7.17
• 3.1.0
• 3.2.3
• 3.3.1
• 3.3.2
• 3.4.0
• 3.4.3

Tested Ruby versions:

• 1.8.7
• 1.9.2

NOTE: The module was tested with Travis with these versions. It may work well on other Puppet or Ruby versions. If that's so, please hit me up.

The following platforms are currently tested (in Docker containers):

If you use it on an other platform, please let me know about it!

Development

Tests

This module use the Voxpupuli tooling, so the Voxpupuli documentation apply about [how to run the test suite=(https://voxpupuli.org/docs/how_to_run_tests/).

Other information

I am open to any pull requests, either for bug fixes or feature developments. I cannot stress the significance of tests sufficiently, so please, write some spec tests and update the documentation as well according to your modification.

Note for commiters:

The master branch is a sacred place, do not commit to it directly, we should touch it only using pull requests.

Reference

Table of Contents

Classes

Public Classes

• syslog_ng: Manage syslog-ng

Private Classes

• syslog_ng::reload: Manage syslog-ng configuration reloading
• syslog_ng::repo: Manage the syslog-ng repository

Defined types

• syslog_ng::config: Add raw configuration
• syslog_ng::destination: Add a destination
• syslog_ng::filter: Add a filter
• syslog_ng::log: Add a log
• syslog_ng::module: Install syslog-ng modules
• syslog_ng::options: Add global options
• syslog_ng::parser: Add a parser
• syslog_ng::rewrite: Add a rewrite rule
• syslog_ng::source: Add a source
• syslog_ng::template: Add a template

Functions

• syslog_ng::generate_log: Gererate log
• syslog_ng::generate_options: Generate options
• syslog_ng::generate_statement: Generate statement

Classes

syslog_ng

The main class of this module. By including it you get an installed syslog-ng with default configuration on your system.

Parameters

The following parameters are available in the syslog_ng class:

• config_file
• package_name
• service_name
• module_prefix
• config_file_header
• package_ensure
• manage_repo
• manage_package
• manage_init_defaults
• modules
• sbin_path
• user
• group
• syntax_check_before_reloads
• init_config_file
• init_config_hash

config_file

Data type: Stdlib::Absolutepath

Configures the path of the configuration file.

package_name

Data type: String[1]

Name of the package to manage when manage_package is true.

service_name

Data type: String[1]

Name of the syslog-ng service.

module_prefix

Data type: String[1]

A string to prepend to syslog-ng module names to obtain the corresponding package names.

config_file_header

Data type: String[1]

A header string that appear on top of the syslog-ng configuration.

package_ensure

Data type: String[1]

The value of the ensure parameter of package resources.

manage_repo

Data type: Boolean

Controls if the module is managing the unofficial repositories of syslog-ng packages. Use true if you want to use the latest version of syslog-ng from the unofficial Debian repository or unofficial RedHat repository.

Default value: false

manage_package

Data type: Boolean

Controls if the module is managing the package resource or not. Use false if you are already handling this in your manifests.

Default value: true

manage_init_defaults

Data type: Boolean

Controls if the module is managing the init script's config file (See init_config_file and init_config_hash).

Default value: false

modules

Data type: Array[String[1]]

Configures additional syslog-ng modules. If manage_package is set to true this will also install the corresponding packages, e.g. syslog-ng-riemann on RedHat if modules = ['riemann'].

Default value: []

sbin_path

Data type: Stdlib::Absolutepath

Configures the path, where syslog-ng and syslog-ng-ctl binaries can be found.

Default value: '/usr/sbin'

user

Data type: String[1]

Configures syslog-ng to run as user.

Default value: 'root'

group

Data type: String[1]

Configures syslog-ng to run as group.

Default value: 'root'

syntax_check_before_reloads

Data type: Boolean

The module always checks the syntax of the generated configuration. If it is not OK, the main configuration (usually /etc/syslog-ng/syslog-ng.conf) will not be overwritten, but you can disable this behavior by setting this parameter to false.

Default value: true

init_config_file

Data type: Stdlib::Absolutepath

Path to the init script configuration file.

init_config_hash

Data type: Hash

Hash of init configuration options to put into init_config_file. This has OS specific defaults which will be merged to user specified value.

Defined types

syslog_ng::config

Some elements of the syslog-ng DSL are not supported by this module (mostly the boolean operators in filters) so you may want to keep some configuration snippets in their original form. This type lets you write texts into the configuration without any parsing or processing.

Every configuration file begins with a @version: line. You can use this type to write this line into the configuration, make comments or use existing snippets.

syslog_ng::config {'version':
   content => '@version: 3.6',
   order => '02'
}

Parameters

The following parameters are available in the syslog_ng::config defined type:

• content
• order

content

Data type: String[1]

Configures the text which must be written into the configuration file. A newline character is automatically appended to its end.

order

Data type: String[1]

Sets the order of this snippet in the configuration file. If you want to write the version line, the order => '02' is suggested, because the auto generated header has order '01'.

Default value: '5'

syslog_ng::destination

Creates a destination in your configuration.

syslog_ng::destination { 'd_udp':
  params => {
    'type' => 'udp',
    'options' => [
      "'127.0.0.1'",
      { 'port' => '1999' },
      { 'localport' => '999' },
    ],
  },
}

#### Parameters

The following parameters are available in the `syslog_ng::destination` defined type:

* [`params`](#-syslog_ng--destination--params)

##### <a name="-syslog_ng--destination--params"></a>`params`

Data type: `Data`

An array of hashes or a single hash.

Default value: `[]`

### <a name="syslog_ng--filter"></a>`syslog_ng::filter`

Creates a filter in your configuration. **It does not support binary operators, such as `and` or `or`**. Please, use a `syslog_ng::config` if you need this functionality.

```puppet
syslog_ng::filter { 'f_tag_filter':
  params => {
    'type' => 'tags',
    'options' => [
      '".classifier.system"',
    ],
  },
}

Parameters

The following parameters are available in the syslog_ng::filter defined type:

• params

params

Data type: Data

An array of hashes or a single hash.

Default value: []

syslog_ng::log

Creates log paths in your configuration. It can create channels, junctions and reference already defined sources, destinations, etc. The syntax is a little bit different then the one previously described under statements.

Here is a simple rule: if you want to reference an already defined type (e.g. s_gsoc2014) use a hash with one key-value pair. The key must be the type of the statement (e.g. source) and the value must be its title.

If you do not specify a reference, use an array instead. Take a look at the example below:

syslog_ng::log { 'l2':
  params => [
    { 'source' => 's_gsoc2014' },
    {
      'junction' => [
        {
          'channel' => [
            { 'filter' => 'f_json' },
            { 'parser' => 'p_json' },
          ],
        },
        {
          'channel' => [
            { 'filter' => 'f_not_json' },
            { 'flags' => 'final' },
          ],
        },
      ],
    },
    { 'destination' => 'd_gsoc' },
  ],
}

Parameters

The following parameters are available in the syslog_ng::log defined type:

• params

params

Data type: Data

The syntax is a bit different, but you can find examples under the tests directory.

Default value: []

syslog_ng::module

Install syslog-ng modules

syslog_ng::options

Creates a global options statement. Currently it is not a class, so you should not declare it multiple times! It is not defined as a class, so you can declare it as other similar types.

syslog_ng::options { "global_options":
  options => {
    'bad_hostname' => "'no'",
    'time_reopen'  => 10,
  },
}

Parameters

The following parameters are available in the syslog_ng::options defined type:

• options

options

Data type: Hash

A hash containing string keys and string values. In the generated configuration the keys will appear in alphabetical order.

Default value: {}

syslog_ng::parser

Creates a parser statement in your configuration.

syslog_ng::parser { 'p_hostname_segmentation':
  params => {
    'type' => 'csv-parser',
    'options' => [
      {
        'columns' => [
          '"HOSTNAME.NAME"',
          '"HOSTNAME.ID"',
        ],
      },
      {'delimiters' => '"-"'},
      {'flags' => 'escape-none'},
      {'template' => '"${HOST}"'},
    ],
  },
}

Parameters

The following parameters are available in the syslog_ng::parser defined type:

• params

params

Data type: Data

An array of hashes or a single hash.

Default value: []

syslog_ng::rewrite

Creates one or more rewrite rules in your configuration.

syslog_ng::rewrite{ 'r_rewrite_subst':
  params => {
    'type' => 'subst',
    'options' => [
      '"IP"',
      '"IP-Address"',
      { 'value' => '"MESSAGE"' },
      { 'flags' => 'global' },
    ],
  },
}

Parameters

The following parameters are available in the syslog_ng::rewrite defined type:

• params

params

Data type: Data

An array of hashes or a single hash.

Default value: []

syslog_ng::source

Creates a source in your configuration.

syslog_ng::source { 's_gsoc':
  params => {
    'type'    => 'tcp',
    'options' => [
      { 'ip' => "'127.0.0.1'" },
      { 'port' => 1999 },
    ],
  },
}

syslog_ng::source { 's_external':
  params => [
    {
      'type'    => 'udp',
      'options' => [
        { 'ip' => ["'127.0.0.1'"] },
        { 'port' => [514] },
      ],
    },
    {
      'type'    => 'tcp',
      'options' => [
        { 'ip' => ["'127.0.0.1'"] },
        { 'port' => [514] },
      ],
    },
    {
      'type'    => 'syslog',
      'options' => [
        { 'flags' => ['no-multi-line', 'no-parse'] },
        { 'ip' => ["'127.0.0.1'"] },
        { 'keep-alive' => ['yes'] },
        { 'keep_hostname' => ['yes'] },
        { 'transport' => ['udp'] },
      ],
    },
  ],
}

Parameters

The following parameters are available in the syslog_ng::source defined type:

• params

params

Data type: Data

An array of hashes or a single hash.

Default value: []

syslog_ng::template

Creates one or more templates in your configuration.

syslog_ng::template {'t_demo_filetemplate':
  params => [
    {
      'type'    => 'template',
      'options' => [
        '"$ISODATE $HOST $MSG\n"',
      ],
    },
    {
      'type'    => 'template_escape',
      'options' => [
        'no',
      ],
    },
  ],
}

Parameters

The following parameters are available in the syslog_ng::template defined type:

• params

params

Data type: Data

An array of hashes or a single hash.

Default value: []

Functions

syslog_ng::generate_log

Type: Ruby 4.x API

Gererate log

syslog_ng::generate_log(Data $options)

Gererate log

Returns: String The generated log

options

Data type: Data

syslog_ng::generate_options

Type: Ruby 4.x API

Generate options

syslog_ng::generate_options(Hash[String[1], Variant[Integer, String]] $options)

Generate options

Returns: String The generated options

options

Data type: Hash[String[1], Variant[Integer, String]]

syslog_ng::generate_statement

Type: Ruby 4.x API

Generate statement

syslog_ng::generate_statement(String $id, String $type, Data $params)

Generate statement

Returns: String The generated statement

id

Data type: String

type

Data type: String

params

Data type: Data

Changelog

All notable changes to this project will be documented in this file. Each new release typically also includes the latest modulesync defaults. These should not affect the functionality of the module.

v4.0.0 (2023-07-04)

Full Changelog

Breaking changes:

• Drop Debian 9 (EOL) #39 (smortex)
• Drop Puppet 6 (EOL) #38 (smortex)
• Drop support of RedHat 6 / Ubuntu 16.04 (EOL) #29 (smortex)

Implemented enhancements:

• Add support for Puppet 8 #40 (smortex)
• Relax dependencies version requirements #37 (smortex)
• Rework documentation #33 (smortex)
• Add support for AlmaLinux #31 (smortex)
• Add support for Rocky #30 (smortex)

Fixed bugs:

• Fix dependencies version bounds #34 (smortex)

Merged pull requests:

• Modernize functions #32 (smortex)
• Fix acceptance tests of expected to fail tests #27 (smortex)

v3.0.0 (2022-03-03)

Full Changelog

Breaking changes:

• Drop support for Debian 8 (EOL) #19 (smortex)
• Drop support for Puppet 5 (EOL) #16 (smortex)
• Drop support for OS which have reached EOL #10 (smortex)
• Drop support for Puppet 4 (EOL) #8 (smortex)

Implemented enhancements:

• Add support for current CentOS releases #21 (smortex)
• Add data types everywhere #20 (smortex)
• Set upper bounds for dependencies #14 (smortex)
• Add support for recent OS #11 (smortex)
• Add support for Puppet 6 and 7 #9 (smortex)
• Switch to GitHub actions #7 (smortex)

Closed issues:

• Tune rubocop Style/TrailingCommaInArguments #25
• Tune rubocop Style/WordArray #24
• Bring back smoke tests #23
• Available module on forge is outdated #5

Merged pull requests:

• Integrate smoke tests as integration tests #26 (smortex)
• Fix module metadata #18 (smortex)
• Add basic acceptance tests #17 (smortex)
• Switch to the new official syslog-ng repo #12 (smortex)
• Remove notice messages #6 (smortex)

v2.2.1 (2020-09-22)

• Add support for SLES
• Add repo management for Debian

v2.1.0 (2018-09-06)

• add structured fact
• fix bug where reload was triggered when service wasn't running
• add datatype

v2.0.0 (2018-03-20)

• Drop support for puppet < 4.9.1
• First official release from ccin2p3 (was ihrwein before)
• Add module data in favor of params.pp

* This Changelog was automatically generated by github_changelog_generator