#pupmod-simp-simp_rsyslog

Table of Contents

• Description
  • This is a SIMP module
• Setup
  • What simp_rsyslog affects
• Usage
  • Local Logging
  • Centralized Logging
  • Log Forwarding
• Reference
• Limitations
• Development
  • Acceptance tests

Description

This module is a SIMP Puppet profile for setting up common Rsyslog configurations as supported by the SIMP ecosystem

This is a SIMP module

This module is a component of the System Integrity Management Platform, a compliance-management framework built on Puppet.

If you find any issues, they may be submitted to our bug tracker.

This module is optimally designed for use within a larger SIMP ecosystem, but it can be used independently:

• When included within the SIMP ecosystem, security compliance settings will be managed from the Puppet server.
• If used independently, all SIMP-managed security subsystems are disabled by default and must be explicitly opted into by administrators. Please review the parameters in simp/simp_options for details.

Setup

What simp_rsyslog affects

This module provides configurations for both Rsyslog local and Rsyslog server configurations.

Usage

Local Logging

To set up local logging, you can simply do the following:

include simp_rsyslog

The $log_collection Hash provides an Rsyslog 7 compatible set of filters that you wish to collect. These will be considered security relevant and fed into /var/log/secure by default.

The Hash has the following format and all entries will be combined with a logical OR.

$log_collection = {
  'programs'   => [ <logged daemon names> ],
  'facilities' => [ <syslog facilities> ],
  'priorities' => [ <syslog priorities> ],
  'msg_starts' => [ <strings the message starts with> ],
  'msg_regex'  => [ <regular expression matches> ]
}

If you need something more complex than this, you will need to configure your own rsyslog rules using the rsyslog::rule defined type.

If you simply want to log EVERYTHING to your remote servers, set simp_rsyslog::collect_everything to true.

If you do this, it is highly recommended that you set simp_rsyslog::log_local to false so that you don't overwhelm your filesystem.

NOTE

If you do not capture the local6 syslog facility, you will lose a lot of SIMP-specific messaging

Centralized Logging

If you wish to collect logs from remote hosts, you can do the following:

Manifest:

include simp_rsyslog

Hieradata:

---
simp_rsyslog::is_server : true

# If your system uses simp/iptables then you should also set the following
iptables::precise_match: true

This will set your system up as an Rsyslog server, using TLS which is capable of collecting both TCP and UDP logs.

At this time, the version of Rsyslog that ships with EL systems cannot handle both TLS and non-TLS TCP connections at the same time. When it can, we will support this mode of log collection.

UDP logs will not be encrypted in transit but are supported for network device compatibility.

Log Forwarding

If you wish to set your system up to forward logs to a set of remote log servers, in either the server or client case, you should use the following in Hiera:

simp_rsyslog::forward_logs: true

This will use the $simp_options::syslog::log_servers and $simp_options::syslog::failover_log_servers variables to set the targets for your logs. Alternatively, you can specify the targets in Hiera directly.

TLS and TCP connections will be used for log forwarding for security purposes.

WARNING

Be VERY careful when setting your simp_rsyslog::log_servers and simp_rsyslog::failover_log_servers Arrays!

There is no foolproof way to detect if you are setting your local log server as part of the Array. If you do this, you may end up with infinite log loops that fill your log server's disk space within minutes.

WARNING

Reference

The module reference can be found in the REFERENCE.md file.

Limitations

This is a SIMP Profile. It will not expose all options of the underlying modules, only the ones that are conducive to a supported SIMP infrastructure.

If you need to do things that this module does not cover, you may need to create your own profile or inherit this profile and extend it to meet your needs.

SIMP Puppet modules are generally intended for use on Red Hat Enterprise Linux and compatible distributions, such as CentOS. Please see the metadata.json file for the most up-to-date list of supported operating systems, Puppet versions, and module dependencies.

Development

Please read our Contribution Guide.

If you find any issues, they can be submitted to our JIRA.

Acceptance tests

This module includes Beaker acceptance tests using the SIMP Beaker Helpers. By default the tests use Vagrant with VirtualBox as a back-end; Vagrant and VirtualBox must both be installed to run these tests without modification. To execute the tests run the following:

bundle install
bundle exec rake beaker:suites

Please refer to the SIMP Beaker Helpers documentation for more information.

Reference

Table of Contents

Classes

• simp_rsyslog: Set up Rsyslog on your system
• simp_rsyslog::forward: Forward Rsyslog logs to remote servers
• simp_rsyslog::local: Set up local Rsyslog logging for the security relevant log files
• simp_rsyslog::server: This class provides a general purpose log server suitable for centralized logging.

Functions

• simp_rsyslog::format_options: Formats a passed log options hash into a form that is appropriate for an Expression Filter-based Rsyslog 7 rule.
• simp_rsyslog::merge_hash_of_arrays: Merges a hash of arrays

Classes

simp_rsyslog

By default, this only sets up the system as a local Rsyslog server with no outside connectivity allowed.

If you set the $is_server parameter, you will set this system up as a log server able to receive input from external systems. Restriction of this input is controlled by the ::rsyslog class and the parameters there should be evaluated if you do not agree with the defaults.

If you include the ::simp_rsyslog::forward class, your system will send its security relevant logs (by default) to the specified $log_servers and $failover_log_servers.

WARNING

Be VERY careful when setting your log_servers and failover_log_servers Arrays!

There is no foolproof way to detect if you are setting your local log server as part of the Array. If you do this, you may end up with infinite log loops that fill your log server's disk space within minutes.

WARNING

This module is a component of the System Integrity Management Platform, a managed security compliance framework built on Puppet.

This module is a SIMP Profile and is not meant to be used outside of the SIMP ecosystem. It may work, but may also require a large number of additional SIMP components to function properly.

• This is a catchall log for security-related messages not written

• to their own logs. Some security logs, such as audit and iptables, will be written to their own logs.

• See also

  • https://simp-project.com
    • SIMP Homepage

Parameters

The following parameters are available in the simp_rsyslog class:

• is_server
• forward_logs
• log_servers
• failover_log_servers
• default_logs
• log_collection
• log_openldap
• log_local
• local_target
• collect_everything
• enable_warning

is_server

Data type: Boolean

Configure the system as a log server for remote hosts

Default value: false

forward_logs

Data type: Boolean

Configure the system to forward the logs specified in the $simp_rsyslog::security_relevant_logs variable

Default value: false

log_servers

Data type: Array[String]

The log servers to which to send remote logs

• If set logs will be sent, in parallel, to all of these servers

Default value: simplib::lookup('simp_options::syslog::log_servers', { 'default_value' => [] })

failover_log_servers

Data type: Array[String]

Failover log servers to use if the primaries go down

Default value: simplib::lookup('simp_options::syslog::failover_log_servers', { 'default_value' => [] })

default_logs

Data type:

Hash[
    Enum[
      'programs',
      'facilities',
      'msg_starts',
      'msg_regex'
    ],
    Array[String]
  ]

The logs that should be forwarded as security relevant to this system.

• All rules will be combined with a logical OR

• If you set this yourself, you will override ALL defaults. If you want to merge in entries, simply use the log_collection parameter.

Default value:

{

    'programs'   => [
      'aide',
      'audispd',
      'audit',
      'auditd',
      'crond',
      'snmpd',
      'sudo',
      'sudosh',
      'systemd',
      'tlog',
      'tlog-rec-session',
      '-tlog-rec-session',
      'yum'
    ],
    'facilities' => [
      '*.emerg',
      'authpriv.*',
      'cron.*',
      'local6.*',
      'local7.warn'
    ],
    # Some versions of rsyslog include the space separator that precedes
    # the message as part of the message body
    'msg_starts' => [' IPT:', 'IPT:', 'IN_99_simp_DROP:', ' IN_99_simp_DROP:'],
    'msg_regex'  => []
  }

log_collection

Data type: Hash[String, Array[String]]

Merge into $default_logs to set the $simp_rsyslog::security_relevant_logs variable.

Options:

• Array[String[1]] Array[String[1]] :programs Logged daemon names: :programs Logged daemon names
• Array[String[1]] Array[String[1]] :facilities Syslog facilities: :facilities Syslog facilities
• Array[String[1]] Array[String[1]] :priorities Syslog priorities: :priorities Syslog priorities
• Array[String[1]] Array[String[1]] :msg_starts Strings the message starts with: :msg_starts Strings the message starts with
• Array[String[1]] Array[String[1]] :msg_regex Regular expression match on the message: :msg_regex Regular expression match on the message

Default value: {}

log_openldap

Data type: Boolean

Collect all OpenLDAP logs

• WARNING these logs are particularly verbose

Default value: false

log_local

Data type: Boolean

Write security-related logs to the filesystem at local_target

Default value: true

local_target

Data type: Stdlib::Absolutepath

Path on the filesystem to which to write security-related logs

Default value: '/var/log/secure'

collect_everything

Data type: Boolean

Set a *.* rule in Rsyslog that matches all logs on the system

• Only applies to forwarded log messages.
• This overrides any other rules that are specified
• This is primarily meant for remote logging where all data is required

Default value: false

enable_warning

Data type: Boolean

By default it will log a warning if a log server is set to forward logs. This can cause a loop unless the simp_rsyslog::servers list does not contain the log server itself.

Default value: true

simp_rsyslog::forward

NOTE: THIS IS A PRIVATE CLASS

Parameters

The following parameters are available in the simp_rsyslog::forward class:

• order
• dest_type
• stop_processing
• permitted_peers

order

Data type: Integer

The shell-glob-based ordering for the rule

Default value: 99

dest_type

Data type: Enum['tcp','udp','relp']

The protocol to use when forwarding to the remote log server

• If you use tcp then you will need to adjust the TLS settings via parameters in the ::rsyslog class directly.

Default value: 'tcp'

stop_processing

Data type: Boolean

Do not continue processing additional Rsyslog rules after the logs have been sent to the remote server.

• In general, you will not want to have this set since you will not have any of the matching logs written to local disk. However, this may be appropriate for ephemeral systems, systems with very slow disks, or systems where you want a minimum of log information to be captured locally.

Default value: false

permitted_peers

Data type: Optional[String]

If TLS is being used, permitted_peers sets the StreamDriverPermittedPeers directive in the forwarding rule actions for the remote rsyslog servers. When undef, the default value computed by rsyslog::rule::remote::stream_driver_permitted_peers is used.

• You will need to set this value if any IP addresses appear in simp_rsyslog::log_servers or simp_rsyslog::failover_servers AND one or more of those servers is not in the same domain as the client.

• StreamDriverPermittedPeers is used to verify servers from the CN, AltDNSname, or fingerprint of the certificate.

• Rsyslog expects a comma separated list. For example: "*.my.domain,server1.my.other.domain"

@see https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfwd.html for more information on how to set this.

Default value: undef

simp_rsyslog::local

NOTE: THIS IS A PRIVATE CLASS

Parameters

The following parameters are available in the simp_rsyslog::local class:

• order

order

Data type: String

The shell-glob-based ordering for the rule

This is currently set to ensure the following:

• Comes after the dynamic local rules that would be on a Rsyslog server (1 and 3 rules from simp_rsyslog::server)
• Comes after the SIMP-module-specific rules that would be on a Rsyslog client (XX* and YY* rules from the sudosh, apache etc. modules).
• Comes before the standard 'ZZ_default.conf' file from SIMP's rsyslog module.

Default value: 'ZZ_0'

simp_rsyslog::server

NOTE: THIS IS A PRIVATE CLASS

It is highly recommended that you look to use the Logstash module at this point.

The following must be set in Hiera for this class to work properly:

rsyslog::global::tls_tcpserver: true

The following are optional for legacy, unencrypted connections.
rsyslog::global::tcpserver: true
rsyslog::global::udpserver: true
rsyslog::global::udpServerAddress: '0.0.0.0'

Loose standard for the rules that will be created:

• 10_default = specific rules to be caught early (ex. matching on programname + error + etc.)
• 11_default = specific rules that have a corresponding "0_default" entry but have a less-specific rule than "0_default" (ex. matching on programname)
• 17_default = catch all for security relevant logs that weren't caught by previous rules
• 19_default = anything else gets sent to messages
• 30_default = stop processing (if appropriate), don't go past this

Parameters

The following parameters are available in the simp_rsyslog::server class:

• server_conf
• process_sudosh_rules
• process_tlog_rules
• process_httpd_rules
• process_dhcpd_rules
• process_snmpd_rules
• process_puppet_agent_rules
• process_puppetserver_rules
• process_auditd_rules
• process_aide_rules
• process_slapd_rules
• process_kern_rules
• process_iptables_rules
• process_firewall_rules
• process_security_relevant_logs
• process_message_rules
• process_mail_rules
• process_cron_rules
• process_emerg_rules
• process_spool_rules
• process_boot_rules
• enable_catchall
• stop_processing
• add_logrotate_rule
• rotate_period
• rotate_preserve
• rotate_size
• logdir
• dyna_key

server_conf

Data type: Optional[String]

The full configuration to use

• Adds the contained rsyslog configuration to the system instead of the default from this module. This allows you complete freedom in specifying your log server ruleset if you do not like the one that is provided. There will be no sanity checking of this string!

Default value: undef

process_sudosh_rules

Data type: Boolean

Enable processing of sudosh rules

Default value: true

process_tlog_rules

Data type: Boolean

Enable processing of tlog rules

Default value: true

process_httpd_rules

Data type: Boolean

Enable processing of httpd rules

Default value: true

process_dhcpd_rules

Data type: Boolean

Enable processing of dhcpd rules

Default value: true

process_snmpd_rules

Data type: Boolean

Enable processing of snmpd rules

Default value: true

process_puppet_agent_rules

Data type: Boolean

Enable processing of puppet agent rules

Default value: true

process_puppetserver_rules

Data type: Boolean

Enable processing of puppetserver rules

Default value: true

process_auditd_rules

Data type: Boolean

Enable processing of auditd rules

Default value: true

process_aide_rules

Data type: Boolean

Enable processing of aide rules

Default value: true

process_slapd_rules

Data type: Boolean

Enable processing of OpenLDAP Server rules

Default value: true

process_kern_rules

Data type: Boolean

Enable processing of kern.* rules

Default value: true

process_iptables_rules

Data type: Boolean

Enable processing of messages starting with IPT: in alignment with the simp/iptables module.

Default value: true

process_firewall_rules

Data type: Boolean

Enable processing of messages starting with IN_99_simp_DROP in alignment with the simp/simp_firewalld module.

Default value: $process_iptables_rules

process_security_relevant_logs

Data type: Boolean

Enable processing of the simp_rsyslog::security_relevant_logs

Default value: true

process_message_rules

Data type: Boolean

Enable the default /var/log/messages traditional processing

Default value: true

process_mail_rules

Data type: Boolean

Enable processing of mail.* rules

Default value: true

process_cron_rules

Data type: Boolean

Enable processing of cron.* rules

Default value: true

process_emerg_rules

Data type: Boolean

Enable processing of *.emerg rules

Default value: true

process_spool_rules

Data type: Boolean

Enable processing of spool.* rules

Default value: true

process_boot_rules

Data type: Boolean

Enable processing of local7.* rules

Default value: true

enable_catchall

Data type: Boolean

Add anything missed by other rules to a catchall.log file

Default value: true

stop_processing

Data type: Boolean

Do not continue processing additional Rsyslog rules after the logs have been sent to the remote server.

• You will probably want to keep this set so that your local system logs are not filled with material from other hosts.

Default value: true

add_logrotate_rule

Data type: Boolean

Add a logrotate rule for the logs that are collected by these server rules

• This will not be applied if you are not using the inbuilt rules since there is no way to know what you are doing.

Default value: true

rotate_period

Data type: Enum['daily','weekly','monthly','yearly']

How often to rotate the local logs

• Has no effect if add_logrotate_rule is false

Default value: 'weekly'

rotate_preserve

Data type: Integer

How many rotated logs to preserve

• 3 months by default

• Has no effect if add_logrotate_rule is false

Default value: 12

rotate_size

Data type: Optional[Integer]

The maximum size of a log file

• $rotate_period will be ignored if this is specified

• Has no effect if add_logrotate_rule is false

Default value: undef

logdir

Data type: Stdlib::AbsolutePath

The directory where the server will send collected logs

Default value: '/var/log/hosts'

dyna_key

Data type: String

The dyna_file rule that organizes the logs as they come in

@see https://www.rsyslog.com/doc/v8-stable/configuration/templates.html @see https://www.rsyslog.com/doc/v8-stable/configuration/properties.html

Default value: '%HOSTNAME%'

Functions

simp_rsyslog::format_options

Type: Ruby 4.x API

Formats a passed log options hash into a form that is appropriate for an Expression Filter-based Rsyslog 7 rule.

simp_rsyslog::format_options(Hash $opts)

Formats a passed log options hash into a form that is appropriate for an Expression Filter-based Rsyslog 7 rule.

Returns: String A formatted entry suitable for injecting into an if statement in Rsyslog 7

opts

Data type: Hash

The options hash.

• All entries will be combined with a logical OR
• NOTE Only the documented Hash keys will be respected

simp_rsyslog::merge_hash_of_arrays

Type: Ruby 4.x API

Merges a hash of arrays

simp_rsyslog::merge_hash_of_arrays(Hash $first_hash, Hash *$additional_hashes_to_merge)

Merges a hash of arrays

Returns: Any Hash Hash containing the superset of keys found in all parameters and merged Array values

first_hash

Data type: Hash

First hash to be merged. Must be a Hash of Arrays.

*additional_hashes_to_merge

Data type: Hash

1 more more additional hashes to be merged. Each must be a Hash of Arrays.

• Mon Oct 23 2023 Steven Pritchard steve@sicura.us - 0.10.0

• [puppetsync] Add EL9 support

• Wed Oct 11 2023 Steven Pritchard steve@sicura.us - 0.9.0

• [puppetsync] Updates for Puppet 8
  • These updates may include the following:
    • Update Gemfile
    • Add support for Puppet 8
    • Drop support for Puppet 6
    • Update module dependencies

• Thu Aug 31 2023 Steven Pritchard steve@sicura.us - 0.8.0

• Add AlmaLinux 8 support

• Mon Jun 12 2023 Chris Tessmer chris.tessmer@onyxpoint.com - 0.7.0

• Add RockyLinux 8 support

• Thu Jun 17 2021 Chris Tessmer chris.tessmer@onyxpoint.com - 0.6.0

• Removed support for Puppet 5
• Ensured support for Puppet 7 in requirements and stdlib

• Tue Feb 02 2021 Liz Nemsick lnemsick.simp@gmail.com - 0.5.2

• Expanded simp/rsyslog dependendency range to < 9.0.0.

• Sat Dec 19 2020 Chris Tessmer chris.tessmer@onyxpoint.com - 0.5.2

• Removed EL6 support

• Wed Oct 28 2020 Trevor Vaughan tvaughan@onyxpoint.com - 0.5.1-0

• Ensure that the docs are correct
• Fix the acceptance tests to use iptables::precise_match
• Remove the filter_IN_99_simp_DROP rules since these are based on the experimental version of simp/simp_firewalld

• Mon Feb 03 2020 Trevor Vaughan tvaughan@onyxpoint.com - 0.5.0-0

• Add EL8 Support
• Add support for firewalld log message collection
• Move logrotate to optional dependencies

• Tue Sep 17 2019 Steven Pritchard steven.pritchard@onyxpoint.com - 0.5.0-0

• Deep merge simp_rsyslog::log_collection

• Thu Jun 06 2019 Steven Pritchard steven.pritchard@onyxpoint.com - 0.4.0-0

• add v2 compliance_markup data
• Add support for Puppet 6
• Add support for puppetlabs-stdlib 6

• Tue Mar 19 2019 Liz Nemsick lnemsick.simp@gmail.com - 0.3.2-0

• Update ELG reference in README.md

• Thu Mar 07 2019 Liz Nemsick lnemsick.simp@gmail.com - 0.3.1-0

• Update the upper bound of stdlib to < 6.0.0
• Update a URL in the README.md

• Wed Oct 17 2018 Jeanne Greulich jeanne.greulich@onyxpoint.com - 0.3.0-0

• Updated acceptance test to run rsyslog twice then check for idempotence.

• Tue Sep 11 2018 Nick Miller nick.miller@onyxpoint.com - 0.3.0-0

• Removed hardcoded strings
  • Make directory where logs are gathered configurable
  • Also make rules that organizes them configurable
• Update systemd fixtures and CI assets
• Add support for Puppet 5 and OEL
• Use iptables module in acceptance tests instead of hand generated rules

• Fri Sep 07 2018 Liz Nemsick lnemsick.simp@gmail.com - 0.3.0-0

• Drop Hiera 4 support

• Mon Aug 27 2018 Jeanne Greulich jeanne.greulich@onyxpoint.com - 0.3.0-0

• Update simp_rsyslog::forward to allow configuration of the StreamDriverPermittedPeers directive in the forwarding rule actions for the remote rsyslog servers. This allows the user to set the correct StreamDriverPermittedPeers value, when the default value is incorrect (e.g., when IP addresses are used in simp_rsyslog::log_servers or simp_rsyslog::failover_servers and one or more of those servers is not in the same domain as the client).

• Mon Aug 27 2018 Trevor Vaughan tvaughan@onyxpoint.com - 0.3.0-0

• Remove redundant rules for sudosh since the puppet module will correctly take care of adding those rules
• Add support for tlog since it will be commonly replacing sudosh across the SIMP infrastructure

• Wed Sep 06 2017 Liz Nemsick lnemsick.simp@gmail.com - 0.2.0-0

• Add processing for aide logs

• Wed Aug 23 2017 Jeanne Greulich jeanne.greulich@onypoint.com - 0.2.0-0

• added processing for snmpd logs

• Tue Aug 01 2017 Trevor Vaughan tvaughan@onyxpoint.com - 0.1.2-0

• Forge release mistake version bump

• Thu Jul 27 2017 Liz Nemsick lnemsick.simp@gmail.com - 0.1.1-0

• Fix bug in rule that ensures local audispd log messages are not duplicated

• Wed May 24 2017 Liz Nemsick lnemsick.simp@gmail.com - 0.1.0-0

• Fix bug whereby audit logs were not being forwarded to remote syslog servers.
• Fix bugs whereby simp_rsyslog::log_collection and simp_rsyslog::log_openldap parameters were overriding simp_rsyslog::default_logs instead of being merged.
• Work around rsyslog inconsistent message parsing behavior that prevented iptables logs from being written to iptables.log and/or being forwarded, for some versions of rsyslog.
• Ensure local audispd log messages are not duplicated.
• Adjust local rule ordering to ensure local sudosh and apache (httpd) log messages are written to their own log files.
• Restore writing of local puppet and puppet-server messages to their own files, by adjusting local security rsyslog rule from simp_rsyslog::local.
• Update puppet requirement in metadata.json

• Wed Apr 19 2017 Nick Markowski nmarkowski@keywcorp.com - 0.1.0-0

• Updated logrotate to use new lastaction API

• Thu Mar 23 2017 Jeanne Greulich jeanne.greulich@onyxpoint.com - 0.0.3

• Fixed path for systemctl

• Fri Feb 24 2017 Jeanne Greulich jeanne.greulich@onyxpoint.com - 0.0.2

• Fixed directory for log rotate of central log server
• add warning if possible log looping detected

• Thu Feb 08 2017 Jeanne Greulich jeanne.greulich@onyxpoint.com - 0.0.1

• Fix path for service to /sbin/service for CentOS 6

• Wed Feb 08 2017 Liz Nemsick lnemsick.simp@gmail.com - 0.0.1

• Fix bug in logrotate rule for simp_rsyslog server

• Tue Dec 27 2016 Trevor Vaughan tvaughan@onyxpoint.com - 0.0.1

• This is the first release of the new simp_rsyslog profile module