Collectd module for Puppet

Description

Puppet module for configuring collectd and plugins.

Usage

The simplest use case is to use all of the configurations in the default collectd.conf file shipped with collectd. This can be done by simply including the class:

include collectd

Collectd is most useful when configured with customized plugins. This is accomplished by removing the default collectd.conf file and replacing it with a file that includes all alternative configurations. Configure a node with the following class declaration:

class { '::collectd':
  purge        => true,
  recurse      => true,
  purge_config => true,
}

Set purge, recurse, and purge_config to true in order to override the default configurations shipped in collectd.conf and use custom configurations stored in conf.d. From here you can set up additional plugins as shown below.

Simple Plugins

Example of how to load plugins with no additional configuration:

collectd::plugin { 'battery': }

where 'battery' is the name of the plugin.

Configurable Plugins

Parameters will vary widely between plugins. See the collectd documentation for each plugin for configurable attributes.

• amqp (see collectd::plugin::amqp below)
• apache (see collectd::plugin::apache below)
• bind (see collectd::plugin::bind below)
• cpu (see collectd::plugin::cpu below)
• csv (see collectd::plugin::csv below)
• curl_json (see collectd::plugin::curl_json below)
• df (see collectd::plugin::df below)
• disk (see collectd::plugin::disk below)
• entropy (see collectd::plugin::entropy below)
• exec (see collectd::plugin::exec below)
• filecount (see collectd::plugin::filecount below)
• interface (see collectd::plugin::interface below)
• iptables (see collectd::plugin::iptables below)
• irq (see collectd::plugin::irq below)
• load (see collectd::plugin::load below)
• libvirt (see collectd::plugin::libvirt below)
• memcached(see collectd::plugin::memcached below )
• memory(see collectd::plugin::memory below )
• mysql (see collectd::plugin::mysql below)
• network (see collectd::plugin::network below)
• nfs (see collectd::plugin::nfs below)
• nginx (see collectd::plugin::nginx below)
• ntpd (see collectd::plugin::ntpd below)
• openvpn (see collectd::plugin::openvpn below)
• ping (see collectd::plugin::ping below)
• postgresql (see collectd::plugin::postgresql below)
• processes (see collectd::plugin:processes below)
• python (see collectd::plugin::python below)
• redis (see collectd::plugin::redis below)
• rrdcached (see collectd::plugin::rrdcached below)
• rrdtool (see collectd::plugin::rrdtool below)
• snmp (see collectd::plugin::snmp below)
• swap (see collectd::plugin::swap below)
• syslog (see collectd::plugin::syslog below)
• tail (see collectd::plugin::tail below)
• tcpconns (see collectd::plugin::tcpconns below)
• unixsock (see collectd::plugin::unixsock below)
• uptime (see collectd::plugin::uptime below)
• users (see collectd::plugin::users below)
• varnish (see collectd::plugin::varnish below)
• vmem (see collectd::plugin::vmem below)
• write_graphite (see collectd::plugin::write_graphite below)
• write_network (see collectd::plugin::write_network below)
• write_riemann (see collectd::plugin::write_riemann below)

####Class: collectd::plugin::amqp

class { 'collectd::plugin::amqp':
  amqphost => '127.0.0.1',
  amqpvhost => 'myvirtualhost',
  graphiteprefix => 'collectdmetrics',
  amqppersistent => true,
}

####Class: collectd::plugin::apache

class { 'collectd::plugin::apache':
  instances => {
    'apache80' => {
      'url' => 'http://localhost/mod_status?auto', 'user' => 'collectd', 'password' => 'hoh2Coo6'
    },
    'lighttpd8080' => {
      'url' => 'http://localhost:8080/mod_status?auto'
    }
  },
}

####Class: collectd::plugin::bind

class { 'collectd::plugin::bind':
  url    => 'http://localhost:8053/',
}

####Class: collectd::plugin::cpu

class { 'collectd::plugin::cpu':
}

####Class: collectd::plugin::csv

class { 'collectd::plugin::csv':
  datadir    => '/etc/collectd/var/lib/collectd/csv',
  storerates => false,
}

####Class: collectd::plugin::curl_json

collectd::plugin::curl_json {
  'rabbitmq_overview':
    url => 'http://localhost:55672/api/overview',
    instance => 'rabbitmq_overview',
    keys => {
      'message_stats/publish' => {'type' => 'gauge'},
    }
}

####Class: collectd::plugin::df

class { 'collectd::plugin::df':
  mountpoints    => ['/u'],
  fstypes        => ['nfs','tmpfs','autofs','gpfs','proc','devpts'],
  ignoreselected => true,
}

####Class: collectd::plugin::disk

class { 'collectd::plugin::disk':
  disks          => ['/^dm/'],
  ignoreselected => true
}

####Class: collectd::plugin::entropy

collectd::plugin::entropy {
}

####Class: collectd::plugin::exec

collectd::plugin::exec {
  'dummy':
    user => nobody,
    group => nogroup,
    exec => ["/bin/echo", "PUTVAL myhost/foo/gauge-flat N:1"],
}

####Class: collectd::plugin::filecount

class { 'collectd::plugin::filecount':
  directories => {
    'active'   => '/var/spool/postfix/active',
    'incoming' => '/var/spool/postfix/incoming'
  },
}

####Class: collectd::plugin::interface

class { 'collectd::plugin::interface':
  interfaces     => ['lo'],
  ignoreselected => true
}

####Class: collectd::plugin::irq

class { 'collectd::plugin::irq':
  irqs           => ['7', '23'],
  ignoreselected => true,
}

####Class: collectd::plugin::iptables

class { 'collectd::plugin::iptables':
  chains  => {
    'nat'    => 'In_SSH',
    'filter' => 'HTTP'
  },
}

####Class: collectd::plugin::load

class { 'collectd::plugin::load':
}

####Class: collectd::plugin::libvirt

The interface_format parameter was introduced in collectd 5.0 and will therefore be ignored (with a warning) when specified with older versions.

class { 'collectd::plugin::libvirt':
  connection       => 'qemu:///system',
  interface_format => 'address'
}

####Class: collectd::plugin::memcached

class { 'collectd::plugin::memcached':
  host => '192.168.122.1',
  port => 11211,
}

####Class: collectd::plugin::memory

class { 'collectd::plugin::memory':
}

####Class: collectd::plugin::mysql

collectd::plugin::mysql::database { 'betadase':
  host        => 'localhost',
  username    => 'stahmna',
  password    => 'secret',
  port        => '3306',
  masterstats => true,
}

####Class: collectd::plugin::network

class { 'collectd::plugin::network':
}

####Class: collectd::plugin::nfs

class { 'collectd::plugin::nfs':
}

####Class: collectd::plugin::nginx

class { 'collectd::plugin::nginx':
  url      => 'https://localhost:8433',
  user     => 'stats',
  password => 'uleePi4A',
}

####Class: collectd::plugin::ntpd

class { 'collectd::plugin::ntpd':
  host           => 'localhost',
  port           => 123,
  reverselookups => false,
  includeunitid  => false,
}

####Class: collectd::plugin::openvpn

class { 'collectd::plugin::openvpn':
  collectindividualusers => false,
  collectusercount       => true,
}

####Class: collectd::plugin::ping

collectd::plugin::ping {
  'example':
    hosts => ['example.com'],
}

####Class: collectd::plugin::postgresql

class { 'collectd::plugin::postgresql':
  databases => {
    'postgres' => {
      'host'     => '/var/run/postgresql/',
      'user'     => 'postgres',
      'password' => 'postgres',
      'sslmode'  => 'disable',
      'query'    => [ 'query_plans', 'queries', 'table_states', 'disk_io' ],
    },
    'devdb' => {
      'host'     => 'host.example.com',
      'port'     => '5432',
      'user'     => 'postgres',
      'password' => 'secret',
      'sslmode'  => 'prefer',
    }
  }
}

####Class: collectd::plugin::processes

class { 'collectd::plugin::processes':
  processes => ['process1', 'process2'],
  process_matches => [
    { name => 'process-all', regex => 'process.*' }
  ],
}

####Class: collectd::plugin::python

collectd::plugin::python {
  'elasticsearch':
    modulepath    => '/usr/lib/collectd',
    module        => 'elasticsearch',
    script_source => 'puppet:///modules/myorg/elasticsearch_collectd_python.py',
    config        => {'Cluster' => 'elasticsearch'},
}

####Class: collectd::plugin::redis

class { 'collectd::plugin::redis':
  nodes => {
    'node1' => {
      'host'     => 'localhost',
    },
    'node2' => {
      'host'     => 'node2.example.com',
      'port'     => '6380',
      'timeout'  => 3000,
    }
  }
}

####Class: collectd::plugin::rrdcached

class { 'collectd::plugin::rrdcached':
  daemonaddress => 'unix:/var/run/rrdcached.sock',
  datadir       => '/var/lib/rrdcached/db/collectd',
}

####Class: collectd::plugin::rrdtool

class { 'collectd::plugin::rrdtool':
  datadir           => '/var/lib/collectd/rrd',
  createfilesasync  => false,
  rrarows           => 1200,
  rratimespan       => [3600, 86400, 604800, 2678400, 31622400],
  xff               => 0.1,
  cacheflush        => 900,
  cachetimeout      => 120,
  writespersecond   => 50
}

####Class: collectd::plugin::snmp

class {'collectd::plugin::snmp':
  data  =>  {
    amavis_incoming_messages => {
      'Type'     => 'counter',
      'Table'    => false,
      'Instance' => 'amavis.inMsgs',
      'Values'   => ['AMAVIS-MIB::inMsgs.0']
    }
  },
  hosts => {
    debianvm => {
      'Address'   => '127.0.0.1',
      'Version'   => 2,
      'Community' => 'public',
      'Collect'   => ['amavis_incoming_messages'],
      'Interval'  => 10
    }
  },
}

####Class: collectd::plugin::swap

class { 'collectd::plugin::swap':
  reportbydevice => false,
  reportbytes    => true
}

####Class: collectd::plugin::syslog

class { 'collectd::plugin::syslog':
  log_level => 'warning'
}

####Class: collectd::plugin::tcpconns

class { 'collectd::plugin::tcpconns':
  localports  => ['25', '12026'],
  remoteports => ['25'],
  listening   => false,
}

####Class: collectd::plugin::tail

collectd::plugin::tail::file { 'exim-log':
  filename => '/var/log/exim4/mainlog',
  instance => 'exim',
  matches  => [
    {
      regex    => 'S=([1-9][0-9]*)',
      dstype   => 'CounterAdd',
      type     => 'ipt_bytes',
      instance => 'total',
    },
    {
      regex    => '\\<R=local_user\\>',
      dstype   => 'CounterInc',
      type     => 'counter',
      instance => 'local_user',
    }
  ]
}

####Class: collectd::plugin::unixsock

class {'collectd::plugin::unixsock':
  socketfile  => '/var/run/collectd-sock',
  socketgroup => 'nagios',
}

####Class: collectd::plugin::uptime

class {'collectd::plugin::uptime':
}

####Class: collectd::plugin::users

class {'collectd::plugin::users':
}

####Class: collectd::plugin::varnish

class { 'collectd::plugin::varnish':
  instances => {
    'instanceName' => {
      'CollectCache' => 'true',
      'CollectBackend' => 'true',
      'CollectConnections' => 'true',
      'CollectSHM' => 'true',
      'CollectESI' => 'false',
      'CollectFetch' => 'true',
      'CollectHCB' => 'false',
      'CollectTotals' => 'true',
      'CollectWorkers' => 'true',
    }
  },
}

####Class: collectd::plugin::vmem

class { 'collectd::plugin::vmem':
  verbose => true,
}

####Class: collectd::plugin::write_graphite

class { 'collectd::plugin::write_graphite':
  graphitehost => 'graphite.examle.org',
}

####Class: collectd::plugin::write_network

Deprecated

class { 'collectd::plugin::write_network':
  servers => {
    'collect1.example.org' => { 'serverport' => '25826' },
    'collect2.example.org' => { 'serverport' => '25826' }
  }
}

####Class: collectd::plugin::write_riemann

class { 'collectd::plugin::write_riemann':
  riemann_host => 'riemann.example.org',
  riemann_port => 5555,
}

##Limitations

This module has been tested on Ubuntu Precise, CentOS 5/6, Solaris 10, and Debian 6/7.

##Development

Running tests

This project contains tests for both rspec-puppet and rspec-system to verify functionality. For in-depth information please see their respective documentation.

Quickstart:

gem install bundler
bundle install
bundle exec rake spec
bundle exec rake spec:system

2014-04-14 Release 2.0.1

Summary:

Fix issue with metadata.json preventing Forge uploads

2014-04-14 Release 2.0.0

Summary:

This release adds support for several new plugins and contains some backwards-incompatible code refactors.

Backwards-incompatible changes:

• Storerates=true is now the default for the Graphite plugin
• Deprecate write_network plugin, use network plugin instead
• Make sure plugin configs are not world readable.
• Handle plugin order with collectd::plugin resource type (all collectd plugins config files will be renumbered from 00-$name.conf to 10-$name.conf)

Features:

• New plugins CSV, uptime, users, entropy, varnish, redis, contextswitch, cpu, nfs, vmem, libvirt
• Add socket parameter to mysql plugin
• Add reportbytes to swap plugin
• Add SeparateInstances setting to write_graphite plugin
• Add metadata.json for Forge search support

Bugs:

• Fix port parameter quotes in memcached plugin
• Fix collectd conf dir path parameter assignment
• Fix processes plugin configuration
• Force ReportReserved to true in df plugin

2014-01-18 Release 1.1.0

Summary:

This release adds support for new plugins and fixes multiple bugs in several plugin configurations.

Features:

• Added load plugin
• Added memory plugin
• Added rrdtool plugin
• Added swap plugin
• Initial version of PostgreSQL plugin
• Add protocol paramter to write_graphite plugin
• add tests for network module
• Support pre-4.7 network configs.

Bugs:

• Fix bug that always creates notification in exec on empty params
• fix typo in apache plugin manifest
• Make sure that plugins are always loaded before their configuration
• Add version check around emitting Protocol config
• mysql: use double quotes according to version
• Allow for alphanumeric collectd version numbers
• (gh-85) Fix null versioncmp bug in plugin templates
• (gh-69) Fix package name on RedHat

2013-12-17 Release 1.0.2

Summary:

This release adds the AMQP plugin and a collectd version fact.

Features:

• Add AMQP plugin
• Add class parameter typesdb
• Use collectd::params::root_group instead of fixed group name
• Add collectd version fact

2013-12-04 Release 1.0.1

Summary:

This release introduces Archlinux osfamily support and support for three new plugins ping, rrdcached, and processes.

Features:

• Initial version of rrdcached plugin
• Add configurable processes plugin
• Add quotes for string values in network plugin
• Add ping plugin
• Add support for Archlinux
• Allow to set all write_graphite options.

Bugs:

• Fixed missing double quotes in unixsock plugin template
• Added comma to syntax error in bind.pp

2013-10-20 Release 1.0.0

Summary:

This release breaks some backwards compatibility on some plugins where they improperly used strings instead of booleans parameters. This release also includes osfamily support for SUSE and FreeBSB and support for four new plugins.

Backwards-incompatible changes:

• Plugins that use to accept strings now use booleans for a more consistent interface across the various plugins
• The main collectd config file now only includes *.conf files to allow plugin specific files to be placed in the conf.d directory.
• The mysql plugin now supports multiple databases via the collectd::plugin::mysql::database define. This change breaks backwards compatiblity on the mysql plugin.

Features:

• osfamily support for SUSE
• osfamily support for FreeBSD
• tail plugin
• exec plugin
• python plugin
• write_riemann plugin

2013-09-27 Release 0.1.0

Summary:

Add curl_json and apache plugin

Backwards-incompatible changes:

• The write_network plugin now accepts a hash of servers

Features:

• Add curl_json plugin
• Added collectd package version parameter
• Add apache plugin

0.0.5 - pdxcat/collectd - 2013/09/24

6adca9a add ntpd plugin ab55fb4 Added tcpconns plugin and example for it in README 677ff72 Adding a new generic write_network plugin define 8cdd8b7 Added memcached plugin and example for it in README c3e4a02 Added examples to README for filecount, SNMP and unixsock f798c84 Added support for file count plugin 66f7392 Added support for SNMP and bumped stdlib dependency version to 3.0.0 7118450 Added support for unix socket plugin a062b11 Fixed source URL 1b214a2 Add testing docs to README

0.0.4 - pdxcat/collectd - 2013/08/26

9f3f0f6 Add bind plugin to readme f052087 Add bind plugin fc0c4c1 Add basic rspec-system test 0b2ec92 Fix README formatting issues 71dcad0 Add configurable plugins to the README 72b9f17 Add test for file_line 7eae468 Add buildstatus to the README b2cb208 Add travis.yml file 8cc16a8 Add purge_config enabled test 26683f0 Add spec test for collectd class 3f30650 Add spec helper files 0bee1d7 Add nginx plugin e2d558a Fix network plugin config syntax 79d8a68 Add network plugin c1085d7 Merge pull request #5 from agenticarus/newer-stdlib 88678bf Depend on newer version of stdlib 7dd9aaa added colorz to the README 628ed78 Added configuration for StoreRates and Port.

0.0.3 - pdxcat/collectd - 2013/05/27

f51660f Fixed test for collectd::plugin::irq 3019516 Added default field in selector for making syntax checkers happy. 39859e7 Added stdlib dependency for file_line

0.0.2 - pdxcat/collectd - 2013/05/09

185e16f Modifications for following puppetlabs style guide and add smoke tests for interface, irq and iptables d68275f Added interface, irq and iptables support. Untested though. b8312cc Added use of scope.lookupvar, allow configuration of threads and option to choose between hostname and fqdn

0.0.1 - pdxcat/collectd - 2013/02/06

8c64104 Add interval and timeout parameters e48288d Add redhat support to the collectd module. b6e3649 Add readme, module, and license files to collectd module. fb6b92f Fix bug in collectd::plugin define 4096cca Add purge_config option to collectd class 2a2974c Add purge option to collectd class 8acbef3 Add syslog plugin 137c919 Add ensure option for collectd::plugin 3406430 Fix plugins with wrong metaparams 315af94 Add collectd::plugin define for unconfigured plugins c085ebc Change include plugin file to include conf.d 171bddd Template openvpn plugin and parameterize the class 09c4df5 Add mysql plugin to collectd. 36f3465 Add disk plugin to collectd. 1e27a29 Fix collectd service subscription to conf file. 15f1ea7 Add Solaris collectd parameters c013e73 Add collectd df plugin support 9cac9c4 Add collectd support.