Forge Home

Modules

Resources

PuppetEcosystemPuppet CommunityUpdates

puppet_metrics_collector

A Puppet module for gathering metrics from PE components
Project URLRSS FeedReport issues

138,044 downloads

1,405 latest version

5.0 quality score

We run a couple of automated
scans to help you access a
module's quality. Each module is
given a score based on how well
the author has formatted their
code and documentation and
modules are also checked for
malware using VirusTotal.

Please note, the information below
is for guidance only and neither of
these methods should be considered
an endorsement by Puppet.

Version information

  • 8.0.0 (latest)
  • 7.2.0
  • 7.1.1
  • 7.1.0
  • 7.0.5
  • 7.0.4
  • 7.0.3
  • 7.0.2
  • 7.0.1
  • 7.0.0
  • 6.6.0
  • 6.5.0
  • 6.4.1
  • 6.4.0
  • 6.3.0
  • 6.2.0
  • 6.1.1
  • 6.1.0
  • 6.0.0
  • 5.3.0
  • 5.2.0
  • 5.1.2
  • 5.1.1
  • 5.1.0
  • 5.0.1
  • 5.0.0
released Mar 26th 2020
This version is compatible with:
  • Puppet Enterprise 2023.6.x, 2023.5.x, 2023.4.x, 2023.3.x, 2023.2.x, 2023.1.x, 2023.0.x, 2021.7.x, 2021.6.x, 2021.5.x, 2021.4.x, 2021.3.x, 2021.2.x, 2021.1.x, 2021.0.x, 2019.8.x, 2019.7.x, 2019.5.x, 2019.4.x, 2019.3.x, 2019.2.x, 2019.1.x, 2019.0.x, 2018.1.x
  • Puppet >= 5.5.1
  • , , , ,

Start using this module

  • r10k or Code Manager
  • Bolt
  • Manual installation
  • Direct download

Add this module to your Puppetfile:

mod 'puppetlabs-puppet_metrics_collector', '6.0.0'
Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add puppetlabs-puppet_metrics_collector
Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install puppetlabs-puppet_metrics_collector --version 6.0.0

Direct download is not typically how you would use a Puppet module to manage your infrastructure, but you may want to download the module in order to inspect the code.

Download

Documentation

puppetlabs/puppet_metrics_collector — version 6.0.0 Mar 26th 2020

Table of Contents

Overview

This module collects metrics provided by the status endpoints of Puppet Enterprise services. The metrics can be used to identify performance issues that may be addressed by performance tuning.

In PE 2018.1.13 and newer and PE 2019.4 and newer, the /metrics/v1 endpoints are disabled by default and access to the /metrics/v2 endpoints are restricted to localhost ... in response to CVE-2020-7943. This module requires access those endpoints to collect additional metrics from PuppetDB, and those metrics will not be collected from remote PuppetDB hosts until these restricted are resolved. Refer to Configuration for Distributed Metrics Collection for a workaround.

Setup

Installation

Install this module with puppet module install puppetlabs-puppet_metrics_collector or add it to your Puppetfile.

To activate this module, classify your Primary Master (aka Master of Masters or MoM) with the puppet_metrics_collector class using your preferred classification method. Below is an example using site.pp.

node 'master.example.com' {
  include puppet_metrics_collector
}

Optionally, you can gather basic system metrics. Unlike service metrics, system metrics have to be enabled locally on each PE Infrastructure Host, and the resulting data will be stored locally on that host. This functionality depends on sysstat.

node 'master.example.com' {
  include puppet_metrics_collector
  include puppet_metrics_collector::system
}

node 'compilerA.example.com', 'compilerB.example.com,' {
  include puppet_metrics_collector::system
}

Note: Do not include the top-level puppet_metrics_collector class on any PE Infrastructure Host other than the Primary Master, otherwise it will collect the same data as the Primary Master.

Configuration

This module automatically configures the hosts it collects metrics from by querying PuppetDB for PE Infrastructure Hosts. If there is an error with the automatic configuration of hosts, refer to Manual Configuration of Hosts.

Parameters

For each Puppet Enterprise service (Puppet Server, PuppetDB, Orchestrator, Ace, Bolt, and ActiveMQ) there are associated <service_name>_ensure, <service_name>_hosts, and <service_name>_port parameters. Refer to manifests/init.pp for details.

output_dir

String: Output directory for collected metrics.

Defaults to /opt/puppetlabs/puppet-metrics-collector.

collection_frequency

Integer: How often to collect metrics, in minutes.

Defaults to 5.

retention_days

Integer: How long to retain collect metrics, in days.

Defaults to 90.

Metrics Server Parameters

The following set of parameters begining with metrics_server_ allows for the specification of a server type to use to generate and (in some cases) send data to a specified metrics server. Currently, both influxdb and graphite types allow for the transfer of data while splunk_hec only generates data.

metrics_server_type

Optional Enum['influxdb','graphite','splunk_hec']: The metrics server type to send data to.

Currently, this module supports influxdb, graphite, and splunk_hec metrics server types.

For the influxdb metrics server type, a metrics_server_db_name must be provided.

For the splunk_hec metrics server type, data cannot be sent to a server, however the command will format the JSON output using the splunk_hec module, which is a requirement for this option. The splunk_hec module can be found on the Forge or GitHub. Setup instructions for the splunk_hec module can be found within that module's README.

metrics_server_hostname

Optional String: The hostname of the metrics server to send data to.

Defaults to undef.

metrics_server_port

Optional Integer: The port number of the metrics server to send data to.

Defaults to undef.

metrics_server_db_name

Optional String: The database name on the metrics server to send data to.

Required for metrics_server_type of influxdb.

Defaults to undef.

override_metrics_command

Optional String: Allows you to define the command that is executed to gather metrics.

Defaults to undef.

Usage

Grepping Metrics

Metrics are formatted as a JSON hash on one line. In order to convert the metric files into a multi-line format, they can be processed with python -m json.tool as per below.

cd /opt/puppetlabs/puppet-metrics-collector
for i in <service_name>/master.example.com/*.json; do echo "$(python -m json.tool < $i)" > $i; done

You can search for useful information by performing a grep, run from inside the directory containing the metrics.

cd /opt/puppetlabs/puppet-metrics-collector
grep <metric_name> <service_name>/master.example.com/*.json

Since the metrics are archived once per day, you can only search metrics for the current day. To search older metrics, decompress the archived files into a subdirectory of /tmp and run your search from inside that directory.

Grepping Puppetserver Metrics

Example:

grep average-free-jrubies puppetserver/master.example.com/*.json

puppetserver/master.example.com/20190404T170501Z.json: "average-free-jrubies": 0.9950009285369501,
puppetserver/master.example.com/20190404T171001Z.json: "average-free-jrubies": 0.9999444653324225,
puppetserver/master.example.com/20190404T171502Z.json: "average-free-jrubies": 0.9999993830655706,

Grepping PuppetDB Metrics

Example:

grep queue_depth puppetdb/master.example.com/*.json

puppetdb/master.example.com/20190404T170501Z.json: "queue_depth": 0,
puppetdb/master.example.com/20190404T171001Z.json: "queue_depth": 0,
puppetdb/master.example.com/20190404T171502Z.json: "queue_depth": 0,

Sharing Metrics Data

When working with Support, you may be asked for an archive of collected metrics data.

This module provides a script, create-metrics-archive to archive metrics data for sending to Support.

/opt/puppetlabs/puppet-metrics-collector/scripts/create-metrics-archive

This script creates the archive in the current working directory.

It takes an optional -m or --metrics-directory parameter (default /opt/puppetlabs/puppet-metrics-collector) to specify an alterate metrics directory to archive.

It takes an optional -r or --retention-days parameter (default: 30) to limit the number of days to include in the archive.

[root@master ~]# /opt/puppetlabs/puppet-metrics-collector/scripts/create-metrics-archive
Created metrics archive: /root/puppet-metrics-collector-20200203T123456Z.tar.gz

Reference

Directory Layout

This module creates an output directory with one subdirectory for each Puppet Enterprise service (Puppet Server, PuppetDB, Orchestrator, Ace, Bolt, and ActiveMQ) that this module has been configured to collect. Each service directory has one subdirectory for each host. Each host directory contains one JSON file, collected every 5 minutes. Once per day, the metrics for each service are archived and compressed.

Example:

/opt/puppetlabs/puppet-metrics-collector/puppetserver
├── master.example.com
│   ├── 20190404T020001Z.json
│   ├── ...
│   ├── 20190404T170501Z.json
│   └── 20190404T171001Z.json
└── puppetserver-2019.04.04.02.00.01.tar.gz
/opt/puppetlabs/puppet-metrics-collector/puppetdb
└── master.example.com
│   ├── 20190404T020001Z.json
│   ├── ...
│   ├── 20190404T170501Z.json
│   ├── 20190404T171001Z.json
└── puppetdb-2019.04.04.02.00.01.tar.gz

Cron Jobs

This module creates two cron jobs for each Puppet Enterprise service:

  • A cron job to collect the metrics
    • Runs as per collection_frequency
  • A cron job to archive collected metrics and delete metrics older than the retention period, as per retention_days
    • Runs at randomly selected time between 12:00 AM and 3:00 AM

Example:

crontab -l
...
# Puppet Name: puppetserver_metrics_collection
*/5 * * * * /opt/puppetlabs/puppet-metrics-collector/scripts/tk_metrics --metrics_type puppetserver --output_dir /opt/puppetlabs/puppet-metrics-collector/puppetserver
# Puppet Name: puppetserver_metrics_tidy
0 2 * * * /opt/puppetlabs/puppet-metrics-collector/scripts/metrics_tidy /opt/puppetlabs/puppet-metrics-collector puppetserver 90

Alternate Setup

Temporary Installation

While a permanent installation is recommended, this module can be temporarily installed with the following commands.

puppet module install puppetlabs-puppet_metrics_collector --modulepath /tmp;
puppet apply -e "class { 'puppet_metrics_collector': }" --modulepath /tmp;

Manual Configuration of Hosts

If necessary, you can manually configure this module by specifying parameters via the class declaration or via Hiera data. The preferred method is via Hiera data. The following examples show you how to specify those parameters for different infrastructures, and assumes you declare this module on the Primary Master.

Monolithic Infrastructure with Compile Masters

Hiera Data Example
puppet_metrics_collector::puppetserver_hosts:
 - 'master.example.com'
 - 'compile-master-1.example.com'
 - 'compile-master-2.example.com'
puppet_metrics_collector::puppetdb_hosts:
 - 'master.example.com'
Class Declaration Example
class { 'puppet_metrics_collector':
  puppetserver_hosts => [
    'master.example.com',
    'compile-master-1.example.com',
    'compile-master-2.example.com'
  ],
  puppetdb_hosts     => ['master.example.com'],
}

Split Infrastructures without Compile Masters

Hiera Data Example
puppet_metrics_collector::puppetserver_hosts:
 - 'split-master.example.com'
puppet_metrics_collector::puppetdb_hosts:
 - 'split-puppetdb.example.com'
Class Declaration Example
class { 'puppet_metrics_collector':
  puppetserver_hosts => ['split-master.example.com'],
  puppetdb_hosts     => ['split-puppetdb.example.com'],
}

Split Infrastructure with Compile Masters

Hiera Data Example
puppet_metrics_collector::puppetserver_hosts:
 - 'split-master.example.com'
 - 'compile-master-1.example.com'
 - 'compile-master-2.example.com'
 puppet_metrics_collector::puppetdb_hosts:
  - 'split-puppetdb.example.com'
Class Definition Example
class { 'puppet_metrics_collector':
  puppetserver_hosts => [
    'split-master.example.com',
    'compile-master-1.example.com',
    'compile-master-2.example.com'
  ],
  puppetdb_hosts => ['split-puppetdb.example.com'],
}

Configuration for Distributed Metrics Collection

This option collect metrics on each PE Infrastructure Host instead of collecting metrics centrally on the Primary Master. This option is discouraged, but allows for the collection of metrics when the Primary Master cannot access the API endpoints of the other PE Infrastructure Hosts. Classify each PE Infrastructure Host with this module, specifying the following parameters.

When classifying a Compile Master, specify these additional parameters:

class { 'puppet_metrics_collector':
  puppetserver_hosts          => ['127.0.0.1'],
  puppetdb_metrics_ensure     => absent,
  orchestrator_metrics_ensure => absent,
  ace_metrics_ensure          => absent,
  bolt_metrics_ensure         => absent,
}

When classifying a PuppetDB Host, specify these additional parameters:

class { 'puppet_metrics_collector':
  puppetdb_hosts              => ['127.0.0.1'],
  puppetserver_metrics_ensure => absent,
  orchestrator_metrics_ensure => absent,
  ace_metrics_ensure          => absent,
  bolt_metrics_ensure         => absent,
}