puppet_metrics_dashboard

• Description
• Setup
  • Upgrade notes
  • Determining where Telegraf runs
  • Requirements
• Usage
  • Configure a Standard Primary Server and a Dashboard node
  • Manual configuration of a complex Puppet Infrastructure
  • Configure Primary Server, Compiler running PuppetDB and a Dashboard node
  • Configure Graphite
  • Configure Telegraf, Graphite, and Archive
  • Allow Telegraf to access PE-PostgreSQL
  • Enable SSL
  • Profile defined types
  • Other possibilities
  • Default dashboards
• Using Archive Metrics
  • Viewing Archive Metrics using Docker
  • Import Archive Metrics into InfluxDB
• Reference
• Limitations
  • Repository failure for InfluxDB packages
  • PostgreSQL metrics collection with older versions of Telegraf
  • Puppet-Telegraf module version 4.0.0
• Development

Description

This module is used to configure Telegraf, InfluxDB, and Grafana, and collect, store, and display metrics collected from Puppet services. By default, those components are installed on a separate Dashboard node by applying the base class of this module to that node. That class will automatically query PuppetDB for Puppet Infrastructure nodes (Primary server, Compilers, PuppetDB hosts, PostgreSQL hosts) or you can specify them via associated class parameters. It is not recommended to apply the base class of this module to one of your Puppet Infrastructure nodes.

You have the option to use the included defined types to configure Telegraf to run on each Puppet Infrastructure node, with the metrics being stored and displayed by another node running InfluxDB and Grafana.
In environments where there is an existing InfluxDB/Grafana installation, this option is recommended. See Determining where Telegraf runs for details.

You have the option of collecting metrics using any or all of the following methods:

• Via Telegraf, which polls Puppet service endpoints (default, recommended)
• Via Puppet Server's built-in Graphite support (Section: Enabling Puppet Server's Graphite support)
• Via Archive files imported from the puppetlabs/puppet_metrics_collector module

Setup

In PuppetDB 6.9.1 & 5.2.13 and newer, the /metrics/v1 endpoints are disabled by default and access to the /metrics/v2 endpoints are restricted to localhost only in response to CVE-2020-7943. Starting with version 2.3.0 of this module, PuppetDB metrics will not be setup by the main class if you are on the versions above or higher unless the main class is applied to the Primary Server. To collect PuppetDB metrics in other scenarios, you should use the puppet_metrics_dashboard::profile::puppetdb class applied to any PuppetDB nodes with the option enable_client_cert => false (the request will be to localhost and doen't require SSL)

Upgrade notes

• Version 2 and up now requires the toml-rb gem installed on the Primary Server and any/all Compilers.
• The puppet_metrics_dashboard::profile::postgres class is deprecated in favor of the puppet_metrics_dashboard::profile::master::postgres_access class.
• Parameters telegraf_agent_interval and http_response_timeout were previously Integers but are now Strings. The value should match a time interval, such as 5s, 10m, or 1h.
• influxdb_urls was previously a String, but is now an Array.

Previous versions of this module added several [[inputs.httpjson]] entries in /etc/telegraf/telegraf.conf. These entries should be removed, as all module-specific settings now reside in individual files within /etc/telegraf/telegraf.d/. Telegraf will continue to work if you do not remove them, however, the old [[inputs.httpjson]] will not be updated going forward.

Determining where Telegraf runs

Telegraf can be configured to run on the Dashboard node, or on each Puppet Infrastructure node. By default, this module configures Telegraf on the Dashboard node by querying PuppetDB to identify each Puppet Infrastructure node. To manually configure Telegraf on the Dashboard node, define the following puppet_metrics_dashboard class parameters: master_list, puppetdb_list and postgres_host_list.

To configure Telegraf to run on each Puppet Infrastructure node, use the corresponding profiles for those nodes. See Profile defined types. Apply the puppet_metrics_dashboard class to the Dashboard node to configure InfluxDB and Grafana, and apply the profile classes on each Puppet Infrastructure node to configure Telegraf.

Requirements

The toml-rb gem is a requirement of the puppet-telegraf module, and needs to be installed in Puppet Server on the Primary Server and any/all Compilers.

Apply the following class to the Primary Server and any/all Compilers to install the gem.

node 'primary.example.com' {
  include puppet_metrics_dashboard::profile::master::install
}
node 'compiler.example.com' {
  include puppet_metrics_dashboard::profile::master::install
}

Or, you can apply the puppet_metrics_dashboard::profile::master::install class to the PE Master Node Group, if using Puppet Enterprise.

Or, you can manually install the gem using the following command.

puppetserver gem install toml-rb

Restart the Puppet Server service after manually installing the gem.

If you are configuring the Dashboard node via a puppet apply workflow, you will need to install the gem into Puppet on that host.

Usage

Configure a Standard Primary Server and a Dashboard node

node 'primary.example.com' {
  include puppet_metrics_dashboard::profile::master::install
  include puppet_metrics_dashboard::profile::master::postgres_access
}

node 'dashboard.example.com' {
  class { 'puppet_metrics_dashboard':
    add_dashboard_examples => true,
    overwrite_dashboards   => false,
  }
}

This will configure Telegraf, InfluxDB, and Grafana on the Dashboard node, and allow Telegraf on that host to access PostgreSQL on the Standard Primary Server.

Note that the add_dashboard_examples parameter enforces state on the example dashboards. Setting the overwrite_dashboards parameter to true disables overwriting your modifications (if any) to the example dashboards.

Manual configuration of a complex Puppet Infrastructure

node 'primary.example.com' {
  include puppet_metrics_dashboard::profile::master::install
}
node 'compiler01.example.com' {
  include puppet_metrics_dashboard::profile::master::install
}
node 'compiler02.example.com' {
  include puppet_metrics_dashboard::profile::master::install
}
node 'postgres01.example.com' {
  include puppet_metrics_dashboard::profile::master::postgres_access
}
node 'postgres02.example.com' {
  include puppet_metrics_dashboard::profile::master::postgres_access
}

node 'dashboard.example.com' {
  class { 'puppet_metrics_dashboard':
    add_dashboard_examples => true,
    overwrite_dashboards   => false,
    configure_telegraf     => true,
    enable_telegraf        => true,
    master_list            => ['primary.example.com', ['compiler01.example.com', 9140], ['compiler02.example.com', 9140]],
    puppetdb_list          => ['puppetdb01.example.com', 'puppetdb02.example.com'],
    postgres_host_list     => ['postgres01.example.com', 'postgres02.example.com'],
  }
}
# Alternate ports are configured using a pair of: [host_name, port_number]

Note that the defaults for this module's class parameters are defined in its data/common.yaml directory.

The *_list parameters can be defined in the class declaration, or elsewhere in Hiera. For example:

puppet_metrics_dashboard::master_list:
  - "primary.example.com"
  - ["compiler01.example.com", 9140]
  - ["compiler02.example.com", 9140]
puppet_metrics_dashboard::puppetdb_list:
  - "puppetdb01.example.com"
  - "puppetdb02.example.com"
puppet_metrics_dashboard::postgres_host_list:
  - "postgres01.example.com"
  - "postgres02.example.com"

Configure Primary Server, Compiler running PuppetDB and a Dashboard node

node 'primary.example.com' {
  include puppet_metrics_dashboard::profile::master::install
}
node 'dbcompiler.example.com' {
  class { 'puppet_metrics_dashboard::profile::dbcompiler::install':
    influxdb_urls => ["http://dashboard.example.com:8086"]
  }
}
node 'dashboard.example.com' {
  class { 'puppet_metrics_dashboard':
    add_dashboard_examples => true,
    overwrite_dashboards   => false,
  }
}

This will configure Telegraf, InfluxDB, and Grafana on the Dashboard node, and allow Telegraf on that host to access PuppetDB on the Compiler running PuppetDB.

Configure Graphite

node 'dashboard.example.com' {
  class { 'puppet_metrics_dashboard':
    add_dashboard_examples => true,
    overwrite_dashboards   => false,
    consume_graphite       => true,
    influxdb_database_name => ['graphite'],
    master_list            => ['primary', 'compiler01'],
  }
}

• This method requires enabling Graphite on the Primary Server and Compilers, as described here. The hostnames that you use in master_list must match the value(s) that you used for metrics_server_id in the puppet_enterprise::profile::master class. You must use hostnames rather than fully-qualified domain names (no dots) both in this class and in the puppet_enterprise::profile::master class.

Configure Telegraf, Graphite, and Archive

Archive refers to files imported from the puppetlabs/puppet_metrics_collector module.

node 'dashboard.example.com' {
  class { 'puppet_metrics_dashboard':
    add_dashboard_examples => true,
    overwrite_dashboards   => false,
    configure_telegraf     => true,
    enable_telegraf        => true,
    consume_graphite       => true,
    influxdb_database_name => ['telegraf', 'graphite', 'puppet_metrics'],
  }
}

Allow Telegraf to access PE-PostgreSQL

The following class is required to be applied to the Primary Server (or the PE Database node if using external PostgreSQL) for collection of PostgreSQL metrics via Telegraf.

node 'primary.example.com' {
  class { 'puppet_metrics_dashboard::profile::master::postgres_access':
    telegraf_host => 'grafana-server.example.com',
  }
}

The telegraf_host parameter is optional. By default, the class will query PuppetDB for Dashboard nodes (with the puppet_metrics_dashboard class applied) and use the certname of the first node in the results. If the PuppetDB lookup fails to find a Dashboard node, and you do not specify telegraf_host then the class outputs a warning.

Refer to Issue 72 if the above generates the following error:

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Function Call, 'versioncmp' parameter 'a' expects a String value, got Undef (file: /opt/puppetlabs/puppet/modules/pe_postgresql/manifests/server/role.pp, line: 66, column: 6) (file: /etc/puppetlabs/code/environments/production/modules/puppet_metrics_dashboard/manifests/profile/master/postgres_access.pp, line: 42) on node primary.example.com

A workaround for that error is to apply the puppet_metrics_dashboard::profile::master::postgres_access class to the PE Database Node Group in the Console, if using Puppet Enterprise.

Enable SSL

node 'dashboard.example.com' {
  class { 'puppet_metrics_dashboard':
    use_dashboard_ssl => true,
  }
}

By default, this will create a set of certificates in /etc/grafana that are based on the Dashboard node's Puppet agent certificates. You can also specify different files by defining the dashboard_cert_file and dashboard_cert_key parameters, but managing certificate content or supplying your own certificates is unsupported by this module.

Note that enabling SSL on Grafana will not allow for running on privileged ports such as 443. To enable that capability, use the suggestions documented in the Grafana documentation

Profile defined types

The module includes defined types that you can use with an existing Grafana implementation. See REFERENCE.md for example usage.

Note that because of the way that the Telegraf module works, these examples will overwrite any configuration in telegraf.conf if it is not already managed by Puppet. See the puppet-telegraf documentation on how to manage that file and add other settings.

Other possibilities

Configure the password for InfluxDB , enable additional TICK Stack components, and customize Grafana.

node 'dashboard.example.com' {
  class { 'puppet_metrics_dashboard':
    influx_db_password  => 'secret',
    enable_chronograf   => true,
    enable_kapacitor    => true,
    grafana_http_port   => 3333,
    grafana_version     => '6.5.2',
  }
}

Default dashboards

This module comes with a few default dashboards. For reference, please check Default Dashboards.

Using Archive Metrics

Viewing Archive Metrics using Docker

This module comes with a the ability to view archive metrics using litmus, Docker, and PDK. This process provisions a Docker container with the module applied, and imports pe_metrics metrics from the puppetlabs-puppet_metrics_collector module. In order to run this, you need Docker and PDK installed on your system.

The following command will provision a local docker container with the local version of this module applied and import the last 30 days of metrics collected by the puppetlabs-puppet_metrics_collector module.

pdk bundle install
pdk bundle exec rake 'viewer[/path/to/offline/metrics]'

The command above will download a CentOS 7 container image configured in the viewer node-set, install Puppet 6.x, copy this module, and apply the module, and import metrics from the specified directory. The UI will be available on http://localhost:3000 with the default admin credentials. This method only uses the pe_metrics database and associated dashboards. To limit the number of days to import metrics, an optional day parameter can be passed into the pdk bundle exec rake 'litmus:viewer[/path/to/offline/metrics,20]' command.

The following command can be used to import additional metrics into a running instance.

pdk bundle exec rake 'viewer:import[/path/to/offline/metrics]'

The following command can be run to destroy the local instance.

pdk bundle exec rake viewer:destroy

Import Archive Metrics into InfluxDB

The json2timeseriesdb script from the puppetlabs/puppet_metrics_collector module can be used to transform its data and import it into InfluxDB.

Examples:

./json2timeseriesdb /opt/puppetlabs/puppet-metrics-collector/puppetserver/*/*.json --convert-to influxdb --influx-db puppet_metrics --netcat dashboard.example.com

This simple example can be used for small number of files. For a large number of files, use --pattern.

./json2timeseriesdb  --pattern '/opt/puppetlabs/puppet-metrics-collector/puppetserver/*/*.json' --convert-to influxdb --influx-db puppet_metrics --netcat dashboard.example.com

The --pattern flag accepts a Ruby glob argument, which the script will internally expand into a list of files.

Reference

This module is documented via pdk bundle exec puppet strings generate --format markdown. Please refer to REFERENCE.md for more information.

Limitations

Repository failure for InfluxDB packages

When installing InfluxDB on CentOS/RedHat 6/7 you may encounter the following error message.

Error: Execution of '/usr/bin/yum -d 0 -e 0 -y install telegraf' returned 1: Error: Cannot retrieve repository metadata (repomd.xml) for repository: influxdb. Please verify its path and try again
Error: /Stage[main]/Puppet_metrics_dashboard::Telegraf/Package[telegraf]/ensure: change from purged to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y install telegraf' returned 1: Error: Cannot retrieve repository metadata (repomd.xml) for repository: influxdb. Please verify its path and try again

This is due to a mismatch in the ciphers available in the operating system and on the InfluxDB repository. To resolve this issue, update nss and curl on the Dashboard node.

yum install curl nss --disablerepo influxdb

PostgreSQL metrics collection with older versions of Telegraf

PostgreSQL metrics collection requires Telegraf version 1.9.1 or later.

Puppet-Telegraf module version 4.0.0

This module is not compatible with puppet-telegraf version 4.0.0. Please use 3.x or 4.1+. See (#158)[https://github.com/puppetlabs/puppet_metrics_dashboard/issues/158] for more details.

Development

Please refer to CONTRIBUTING.md for more information.

Reference

Table of Contents

Classes

Public Classes

• puppet_metrics_dashboard: Installs and configures a stack for collecting, storing, and displaying Puppet Infrastructure metrics
• puppet_metrics_dashboard::profile::dbcompiler::install: Apply this class to compilers running PuppetDB to configure Telegraf and collect puppetserver and puppetdb metrics
• puppet_metrics_dashboard::profile::ldap_auth: Apply this class to your dashboard node after enabling LDAP authentication
• puppet_metrics_dashboard::profile::master::install: Install requirements for the voxpupuli/puppet-telegraf module
• puppet_metrics_dashboard::profile::master::postgres_access: Apply this class to a PE PostgreSQL node to allow access by Telegraf.
• puppet_metrics_dashboard::profile::postgres: This class is deprecated. Please use the Puppet_metrics_dashboard::Profile::Master::Postgres_access class.

Private Classes

• puppet_metrics_dashboard::config: Configures dashboard components.
• puppet_metrics_dashboard::dashboards: Configures Grafana dashboards.
• puppet_metrics_dashboard::dashboards::graphite: Installs graphite example dashboards
• puppet_metrics_dashboard::dashboards::puppet_metrics: Installs puppet_metrics example dashboards
• puppet_metrics_dashboard::dashboards::telegraf: Installs telegraf example dashboards
• puppet_metrics_dashboard::grafana: Install and configure Grafana
• puppet_metrics_dashboard::install: Installs InfluxDB components.
• puppet_metrics_dashboard::post_start_configs: InfluxDB post-start configs
• puppet_metrics_dashboard::repos: Configures InfluxDB and Grafana repos
• puppet_metrics_dashboard::service: Manages services
• puppet_metrics_dashboard::telegraf: Install and configure Telegraf
• puppet_metrics_dashboard::telegraf::config: Configures Telegraf
• puppet_metrics_dashboard::telegraf::service: Manages the Telegraf service

Defined types

• puppet_metrics_dashboard::certs: Copy Puppet Agent keypair for use by metric services.
• puppet_metrics_dashboard::profile::compiler: Apply this class to a Primary Server or Compiler to collect puppetserver metrics
• puppet_metrics_dashboard::profile::master::postgres: Apply this class to an agent running pe-postgresql to collect postgres metrics
• puppet_metrics_dashboard::profile::puppetdb: Apply this class to a node running puppetdb to collect puppetdb metrics

Functions

• puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile
• puppet_metrics_dashboard::puppetdb_metrics: function used to determine the set of needed PuppetDB metrics based on PE version
• puppet_metrics_dashboard::puppetdb_v2_metrics

Data types

• Puppet_metrics_dashboard::HostList: A list of hostnames, or pairs of hostname and port.
• Puppet_metrics_dashboard::Puppetdb_metric: A metric name corresponding to an endpoint (url)

Classes

puppet_metrics_dashboard

Installs and configures a stack for collecting, storing, and displaying Puppet Infrastructure metrics. Refer to data/common.yaml for additional parameter defaults.

Examples

Grafana with no login

class { 'puppet_metrics_dashboard':
  grafana_config => {
    'users'          => {
      'allow_sign_up' => false,
    },
    'auth.anonymous' => {
      'enabled' => true,
    },
  },
}

Configure Telegraf to collect metrics from a list of Primary Server, Compilers, PuppetDB, and PostgreSQL servers

class { 'puppet_metrics_dashboard':
  add_dashboard_examples => true,
  overwrite_dashboards   => false,
  configure_telegraf     => true,
  enable_telegraf        => true,
  master_list            => ['primary.example.com', ['compiler01.example.com', 9140], ['compiler02.example.com', 9140]],
  puppetdb_list          => ['puppetdb01.example.com', 'puppetdb02.example.com'],
  postgres_host_list     => ['postgres01.example.com', 'postgres02.example.com'],
}

Configure Graphite to accept metrics from a list of Primary Server and Compilers

class { 'puppet_metrics_dashboard':
  add_dashboard_examples => true,
  overwrite_dashboards   => false,
  consume_graphite       => true,
  influxdb_database_name => ['graphite'],
  master_list            => ['primary', 'compiler01'],
}

Configure Telegraf, Graphite, and Archive

class { 'puppet_metrics_dashboard':
  add_dashboard_examples => true,
  overwrite_dashboards   => false,
  consume_graphite       => true,
  configure_telegraf     => true,
  enable_telegraf        => true,
  influxdb_database_name => ['telegraf', 'graphite', 'puppet_metrics'],
}

Parameters

The following parameters are available in the puppet_metrics_dashboard class:

• manage_repos
• add_dashboard_examples
• overwrite_dashboards
• enable_chronograf
• enable_kapacitor
• enable_telegraf
• configure_telegraf
• consume_graphite
• influxdb_database_name
• influxdb_urls
• influx_db_service_name
• influx_db_password
• influx_archive_source
• telegraf_db_name
• telegraf_db_retention_duration
• http_response_timeout
• telegraf_agent_interval
• pg_query_interval
• use_dashboard_ssl
• tidy_telegraf_configs
• dashboard_cert_file
• dashboard_cert_key
• enable_ldap_auth
• grafana_http_port
• grafana_password
• grafana_version
• overwrite_dashboards_file
• grafana_config
• grafana_manage_repo
• grafana_archive_source
• grafana_install_method
• master_list
• puppetdb_list
• postgres_host_list
• puppetdb_metrics

manage_repos

Data type: Boolean

Whether to configure apt / yum repositories for required packages.

add_dashboard_examples

Data type: Boolean

Whether to add the example Grafana dashboards for the configured InfluxDB databases. Defaults to false. Note: These dashboards are managed and any changes will be overwritten unless the overwrite_dashboards is set to false.

overwrite_dashboards

Data type: Boolean

Whether to overwrite the example Grafana dashboards. Defaults to true This parameter disables overwriting the example Grafana dashboards. It takes effect after the second Puppet run, and populates a overwrite_dashboards_disabled fact. Only used when add_dashboard_examples is true.

enable_chronograf

Data type: Boolean

Whether to install chronograf. Defaults to false No configuration of chronograf is included at this time.

enable_kapacitor

Data type: Boolean

Whether to install kapacitor. Defaults to false No configuration of kapacitor is included at this time.

enable_telegraf

Data type: Boolean

Whether to install telegraf. Defaults to true No configuration is done unless configure_telegraf is set to true.

configure_telegraf

Data type: Boolean

Whether to configure the Telegraf service. Defaults to true This parameter enables and configures Telegraf to query the *_list hosts for metrics. Metrics will be stored in the telegraf database in InfluxDb. Ensure that influxdb_database_name contains telegraf when using this parameter. Only used when enable_telegraf is true.

consume_graphite

Data type: Boolean

Whether to enable the InfluxDB Graphite plugin. Defaults to false This parameter enables the Graphite plugin for InfluxDB to allow for consuming Graphite metrics. Ensure influxdb_database_name contains graphite when using this parameter. Note: To consume metrics sent from Puppet Server, this must to be set to true.

influxdb_database_name

Data type: Array[String]

An Array of databases that should be created in InfluxDB. Valid values are telegraf, graphite, puppet_metrics, and any other string. Defaults to ["telegraf"] Each database in the array will be created in InfluxDB. telegraf, graphite, and puppet_metrics are specially named and will be used with their associated metric collection method. Any other database name will be created, but not associated with components in this module.

influxdb_urls

Data type: Array[String]

An Array containing urls defining InfluxDB instances for Telegraf.

influx_db_service_name

Data type: String

Name of the InfluxDB service used by the operating system.

influx_db_password

Data type: String

The password for the InfluxDB admin user. Defaults to puppet

influx_archive_source

Data type: Optional[Stdlib::Httpsurl]

URL if you want to install influx from tar.gz file

Default value: undef

telegraf_db_name

Data type: String

The InfluxDB database where Telegraf metrics are stored.

telegraf_db_retention_duration

Data type: Optional[String[1]]

The retention duration for database used for Telegraf metrics. Defaults to undef. Specify a value to setup retention duration. Example: 4w is 4 weeks. To update a new duration, set it as undef first and run puppet agent to reset. Then, specify the new value.

Default value: undef

http_response_timeout

Data type: String[2]

Timeout for Telegraf HTTP requests. Defaults to 5s

telegraf_agent_interval

Data type: String[2]

Frequency of Telegraf HTTP queries for metrics. Defaults to 5s

pg_query_interval

Data type: String[2]

Frequency of Telegraf PostgreSQL queries for metrics. Defaults to 10m

use_dashboard_ssl

Data type: Boolean

Whether to enable SSL in Grafana. Valid values are true, false. Defaults to false

tidy_telegraf_configs

Data type: Boolean

Whether or not to remove unmanaged configuration files from /etc/telegraf/telegraf.d. Defaults to false

dashboard_cert_file

Data type: String

The location of the Grafana certficiate. Defaults to /etc/grafana/${clientcert}_cert.pem Only used when use_dashboard_ssl is true.

dashboard_cert_key

Data type: String

The location of the Grafana private key. Defaults to /etc/grafana/${clientcert}_key.pem Only used when use_dashboard_ssl is true.

enable_ldap_auth

Data type: Boolean

Whether to enable LDAP authentication in Grafana Valid values are true, false. Defaults to false

grafana_http_port

Data type: Integer

The port for the Grafana web interface. Valid values are Integers from 1024 to 65536. Defaults to 3000 This should be a nonprivileged port (above 1024).

grafana_password

Data type: String

The password for the Grafana admin user. Defaults to admin

grafana_version

Data type: String

The version of Grafana to install. Valid values are String versions of Grafana.

overwrite_dashboards_file

Data type: String

File used to populate the overwrite_dashboards fact.

grafana_config

Data type: Hash

Hash of arbitrary configuration settings to pass to Grafana. These are added to grafana.ini with top-level keys becoming sections and their key-value children becoming settings.

grafana_manage_repo

Data type: Boolean

Whether to configure apt / yum repositories for grafana packages.

Default value: $manage_repos

grafana_archive_source

Data type: Optional[Stdlib::Httpsurl]

URL if you want to install grafana from tar.gz file

Default value: undef

grafana_install_method

Data type: Enum['docker', 'archive', 'package', 'repo']

grafana module allows to specify the installation method. Set to 'archive' to install Grafana using the tar archive. Set to 'docker' to install Grafana using the official Docker container. Set to 'package' to install Grafana using .deb or .rpm packages. Set to 'repo' to install Grafana using an apt or yum repository. Defaults to 'package'.

Default value: 'repo'

master_list

Data type: Puppet_metrics_dashboard::HostList

An Array of servers that Telegraf will collect Puppet Server metrics from. Entries may be:

• A String that contains a hostname or IP address. (The module will use a default port number of 8140)
• An Array where the first entry is a String that contains a hostname or IP address, and the second entry is an Integer that specifies the port number. Defaults to the result of a PuppetDB query, or [$trusted['certname']]

Default value: puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile('master')

puppetdb_list

Data type: Puppet_metrics_dashboard::HostList

An Array of servers that Telegraf will collect PuppetDB metrics from. Entries may be:

• A String that contains a hostname or IP address. (The module will use a default port number of 8081)
• An Array where the first entry is a String that contains a hostname or IP address, and the second entry is an Integer that specifies the port number. Defaults to the result of a PuppetDB query, or [$trusted['certname']]

Default value: puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile('puppetdb')

postgres_host_list

Data type: Puppet_metrics_dashboard::HostList

An Array of servers that Telegraf will collect PostgreSQL metrics from. Entries may be:

• A String that contains a hostname or IP address. (The module will use a default port number of 5432)
• An Array where the first entry is a String that contains a hostname or IP address, and the second entry is an Integer that specifies the port number. Defaults to the results of a PuppetDB query, or [$trusted['certname']]

Default value: puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile('database')

puppetdb_metrics

Data type: Puppet_metrics_dashboard::Puppetdb_metric

An Array of Hashes containing name/url pairs for each PuppetDB metric. Refer to functions/puppetdb_metrics.pp for defaults.

Default value: puppet_metrics_dashboard::puppetdb_metrics()

puppet_metrics_dashboard::profile::dbcompiler::install

Apply this class to compilers running PuppetDB to configure Telegraf and collect puppetserver and puppetdb metrics

Parameters

The following parameters are available in the puppet_metrics_dashboard::profile::dbcompiler::install class:

• influxdb_urls
• timeout
• compiler
• puppetdb_host
• cm_port
• db_port
• interval
• tidy_telegraf_configs
• manage_repo
• puppetdb_metrics

influxdb_urls

Data type: Array[String]

An Array containing urls defining InfluxDB instances for Telegraf.

timeout

Data type: String[2]

Default timeout of http calls. Defaults to 5 seconds

Default value: lookup('puppet_metrics_dashboard::http_response_timeout')

compiler

Data type: Variant[String,Tuple[String, Integer]]

The FQDN of the compiler / master. Defaults to the FQDN of the server where the profile is applied

Default value: $facts['networking']['fqdn']

puppetdb_host

Data type: Variant[String,Tuple[String, Integer]]

Where to query the puppetdb host. Defaults to localhost.

Default value: 'localhost'

cm_port

Data type: Integer[1]

The port that the puppetserver service listens on on your compiler. Defaults to 8140

Default value: 8140

db_port

Data type: Integer[1]

The port that the puppetdb service listens on on your compiler. Defaults to 8081

Default value: 8081

interval

Data type: String[2]

The frequency that telegraf will poll puppetserver metrics. Defaults to '5s'

Default value: '5s'

tidy_telegraf_configs

Data type: Boolean

Whether or not to remove unmanaged configuration files from /etc/telegraf/telegraf.d. Defaults to false.

Default value: lookup('puppet_metrics_dashboard::tidy_telegraf_configs')

manage_repo

Data type: Optional[Boolean]

Boolean. Whether or not to manage Telegraf's repo. Defaults to true

Default value: true

puppetdb_metrics

Data type: Puppet_metrics_dashboard::Puppetdb_metric

Default value: puppet_metrics_dashboard::puppetdb_metrics()

puppet_metrics_dashboard::profile::ldap_auth

Server Parameters

If your LDAP server does not support the memberOf attribute add these options:

Server Attributes Parameters: Specify names of the LDAP attributes your LDAP uses.

Group Mapping Parameters

Parameters

The following parameters are available in the puppet_metrics_dashboard::profile::ldap_auth class:

• ldap_host
• ldap_port
• ldap_ssl
• ldap_tls
• ldap_ssl_skip
• ldap_bind_dn
• ldap_bind_password
• ldap_search_filter
• ldap_search_base
• ldap_groupsearch_filter
• ldap_groupsearch_base
• ldap_groupsearch_user_attr
• ldap_name
• ldap_surname
• ldap_username
• ldap_member_of
• ldap_email
• ldap_group_dn
• ldap_grafana_role

ldap_host

Data type: String

FQDN or IP of the LDAP server (specify multiple hosts space separated).

ldap_port

Data type: Integer

The port the LDAP service is listening on. Defaults to non-secure port 389.

Default value: 389

ldap_ssl

Data type: Boolean

Enable the use of SSL. Defaults to the false :: Set to true if LDAP server supports TLS

Default value: false

ldap_tls

Data type: Boolean

Set to true to connect LDAP server with STARTTLS pattern. Defaults to false.

Default value: false

ldap_ssl_skip

Data type: Boolean

Set to false if you do not want to skip SSL cert validation. Defaults to true.

Default value: true

ldap_bind_dn

Data type: String

Search user bind dn. If you can provide a single bind expression that matches all possible users, you can skip specifying ldap_bind_password.

ldap_bind_password

Data type: Optional[String]

Search user bind password. If the password contains # or ; you have to wrap it with triple quotes. Ex """#password;"""

Default value: null

ldap_search_filter

Data type: String

User search filter, for example "(cn=%s)" or "(sAMAccountName=%s)" or "(uid=%s)"

ldap_search_base

Data type: String

An array of base dns to search through

ldap_groupsearch_filter

Data type: Optional[String]

Group search filter, to retrieve the groups of which the user is a member (only set if memberOf attribute is not available)

Default value: null

ldap_groupsearch_base

Data type: Optional[String]

An array of the base DNs to search through for groups.

Default value: null

ldap_groupsearch_user_attr

Data type: Optional[String]

The %s in ldap_groupsearch_filter will be replaced with the attribute defined here.

Default value: null

ldap_name

Data type: Optional[String]

Default value: name

ldap_surname

Data type: Optional[String]

Default value: surname

ldap_username

Data type: Optional[String]

Default value: username

ldap_member_of

Data type: Optional[String]

Default value: memberOf

ldap_email

Data type: Optional[String]

Default value: email

ldap_group_dn

Data type: String

Ldap DN of LDAP group. Will accept wildcard to match all groups "*".

ldap_grafana_role

Data type: String

Assign users one of the following roles: "Admin", "Editor" or "Viewer". Defaults to "Admin".

Default value: 'Admin'

puppet_metrics_dashboard::profile::master::install

Install requirements for the voxpupuli/puppet-telegraf module.

Examples

Apply this class to the Primary Server and any/all Compilers

include puppet_metrics_dashboard::profile::master::install

Parameters

The following parameters are available in the puppet_metrics_dashboard::profile::master::install class:

• manage_ldap_auth

manage_ldap_auth

Data type: Boolean

Default value: true

puppet_metrics_dashboard::profile::master::postgres_access

Apply this class to a PE PostgreSQL node to allow access by Telegraf.

Examples

Apply this class to PE PostgreSQL nodes

class { 'puppet_metrics_dashboard::profile::master::postgres_access':
  telegraf_host => 'dashboard.example.com',
}

Parameters

The following parameters are available in the puppet_metrics_dashboard::profile::master::postgres_access class:

• telegraf_host

telegraf_host

Data type: Optional[String[1]]

The FQDN of the host running Telegraf. Defaults to an empty string. You can define this parameter, otherwise this class will query PuppetDB for a dashboard host.

Default value: undef

puppet_metrics_dashboard::profile::postgres

This class is deprecated. Please use the Puppet_metrics_dashboard::Profile::Master::Postgres_access class.

Parameters

The following parameters are available in the puppet_metrics_dashboard::profile::postgres class:

• grafana_host

grafana_host

Data type: String

The FQDN of the host where telegraf runs. Defaults to an empty string. You can explicitly set this parameter or the class attempts to lookup which host has the puppet_metrics_dashboard class applied in PuppetDB. If the parameter is not set and the lookup does not return anything we issue a warning.

Default value: ''

Defined types

puppet_metrics_dashboard::certs

This type creates copies of the Puppet Agent's SSL keypair in /etc/${service} with user+group ownership set to ${service}. These certificates are used when configuring Grafana to use SSL and to connect Telegraf with PE Services.

Parameters

The following parameters are available in the puppet_metrics_dashboard::certs defined type:

• service
• ssl_dir

service

Data type: Any

The service name to associate with the keypair copy.

Default value: $name

ssl_dir

Data type: Any

The directory to copy Puppet Agent SSL files from. Defaults to the value of puppet config print --section server ssldir used by the Puppet Server, often /etc/puppetlabs/puppet/ssl. Use Hiera to override this value if agents have a different ssldir setting or if bolt apply is being used.

Default value: lookup('puppet_metrics_dashboard::certs::ssl_dir', {default_value => $settings::ssldir})

puppet_metrics_dashboard::profile::compiler

Apply this class to a Primary Server or Compiler to collect puppetserver metrics

Examples

Add telegraf to a Primary Server / Compiler

puppet_metrics_dashboard::profile::compiler{ $facts['networking']['fqdn']:
  timeout => '5s',
}

Parameters

The following parameters are available in the puppet_metrics_dashboard::profile::compiler defined type:

• timeout
• compiler
• port
• interval

timeout

Data type: String[2]

Default timeout of http calls. Defaults to 5 seconds

Default value: lookup('puppet_metrics_dashboard::http_response_timeout')

compiler

Data type: Variant[String,Tuple[String, Integer]]

The FQDN of the Compiler / Primary Server. Defaults to the FQDN of the server where the profile is applied

Default value: $facts['networking']['fqdn']

port

Data type: Integer[1]

The port that the puppetserver service listens on on your compiler. Defaults to 8140

Default value: 8140

interval

Data type: String[2]

The frequency that telegraf will poll for metrics. Defaults to '5s'

Default value: '5s'

puppet_metrics_dashboard::profile::master::postgres

Apply this class to an agent running pe-postgresql to collect postgres metrics

Examples

Add telegraf to a postgres server

puppet_metrics_dashboard::profile::master::postgres{ $facts['networking']['fqdn']:
  query_interval => '10m',
}

Parameters

The following parameters are available in the puppet_metrics_dashboard::profile::master::postgres defined type:

• query_interval
• postgres_host
• port
• databases

query_interval

Data type: String[2]

How often to run the queries in minutes. Defaults to 10 minutes.

Default value: lookup('puppet_metrics_dashboard::pg_query_interval')

postgres_host

Data type: Variant[String,Tuple[String, Integer]]

The FQDN of the postgres host. Defaults to the FQDN of the server where the profile is applied

Default value: $facts['networking']['fqdn']

port

Data type: Integer[1]

The port that the postgres service listens on. Defaults to 5432

Default value: 5432

databases

Data type: Array[String[1]]

An Array of databases to query on. Defaults to ['pe-puppetdb','pe-rbac','pe-activity','pe-classifier']

Default value: ['pe-puppetdb','pe-rbac','pe-activity','pe-classifier']

puppet_metrics_dashboard::profile::puppetdb

Apply this class to a node running puppetdb to collect puppetdb metrics

Examples

Add telegraf to a puppetdb node

puppet_metrics_dashboard::profile::puppetdb{ $facts['networking']['fqdn']:
  timeout          => '5s',
  puppetdb_metrics => puppet_metrics_dashboard::puppetdb_metrics(), # this is the default value
}

Parameters

The following parameters are available in the puppet_metrics_dashboard::profile::puppetdb defined type:

• timeout
• puppetdb_metrics
• puppetdb_host
• port
• interval
• enable_client_cert

timeout

Data type: String[2]

Default timeout of http calls. Defaults to 5 seconds

Default value: lookup('puppet_metrics_dashboard::http_response_timeout')

puppetdb_metrics

Data type: Puppet_metrics_dashboard::Puppetdb_metric

An array of hashes containing name / url pairs for each puppetdb metric. See functions/puppetdb_metrics.pp for defaults.

Default value: puppet_metrics_dashboard::puppetdb_metrics()

puppetdb_host

Data type: Variant[String,Tuple[String, Integer]]

Where to query the puppetdb host. Defaults to localhost.

Default value: 'localhost'

port

Data type: Integer[1]

The port that the puppetdb service listens on on your compiler. Defaults to 8081

Default value: 8081

interval

Data type: String[2]

The frequency that telegraf will poll for metrics. Defaults to '5s'

Default value: '5s'

enable_client_cert

Data type: Boolean

A boolean to enable using the client certificate for the PuppetDB queries. Defaults to true

Default value: true

Functions

puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile

Type: Puppet Language

The puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile function.

puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile(String $profile)

The puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile function.

Returns: Array

profile

Data type: String

puppet_metrics_dashboard::puppetdb_metrics

Type: Puppet Language

The list of metrics to pull from PuppetDB depends on the PE version. To avoid having a data file for each version we utilize this function to build the needed array of hashes.

puppet_metrics_dashboard::puppetdb_metrics()

The list of metrics to pull from PuppetDB depends on the PE version. To avoid having a data file for each version we utilize this function to build the needed array of hashes.

Returns: Array[Hash] An array of hashes containing name / url pairs for each puppetdb metric.

puppet_metrics_dashboard::puppetdb_v2_metrics

Type: Puppet Language

The puppet_metrics_dashboard::puppetdb_v2_metrics function.

puppet_metrics_dashboard::puppetdb_v2_metrics()

The puppet_metrics_dashboard::puppetdb_v2_metrics function.

Returns: Boolean

Data types

Puppet_metrics_dashboard::HostList

A list of hostnames, or pairs of hostname and port.

Alias of

Array[Variant[
    String,
    Tuple[String, Integer]
  ]]

Puppet_metrics_dashboard::Puppetdb_metric

A metric name corresponding to an endpoint (url)

Alias of

Tuple[Struct[{
    name => String[1],
    url  => String[1]
  }], 1, default]

Change log

All notable changes to this project will be documented in this file. The format is based on Keep a Changelog and this project adheres to Semantic Versioning.

v2.7.0 (2022-02-24)

Full Changelog

Added

• Allow ssl_dir to be configured via Hiera #205 (Sharpie)
• Enable PuppetDB SSL in dbcompiler profile #203 (Sharpie)

Fixed

• Accept 503 responses from PE /status APIs #204 (Sharpie)

v2.6.1 (2021-11-04)

Full Changelog

Fixed

• (SUP-2779) Fix the bug - telegraf postgres metrics unable to obtain data #200 (henrywangpuppet)

v2.6.0 (2021-11-01)

Full Changelog

Added

• (SUP-2769) Bump default Grafana version from 5.1.4 to 8.2.2 #197 (suckatrash)
• (Sup-2754) Adding SLES support #194 (m0dular)
• (SUP-2194) To add datasource variable for Archive Dashboards - PR 185 Replacement #192 (henrywangpuppet)
• Added repo_gpgcheck and single quotes to influxdb #189 (di2ejenkins)
• (SUP-2137) To add parameters to configure telegraf database retention policy #188 (henrywangpuppet)
• (GH-158) Add support for puppet-telegraf 4.1+ #166 (jarretlavallee)
• (#151) Install toml gem by default. #154 (coreymbe)

Fixed

• (SUP-2732) Fix PDB data tagged onto dashboard server issue and templa… #191 (henrywangpuppet)
• (GH-172) Add cgroup mapping for viewer #173 (jarretlavallee)

v2.5.0 (2021-05-27)

Full Changelog

Added

• Enable remote PuppetDB collection in 2019.8.5+ #139 (jarretlavallee)
• (SUP-2195) Add Archive Postgres Performance dashboard #128 (Sharpie)
• Enable LDAP authentication for Grafana #108 (coreymbe)

Fixed

• Remove slash from dashboard name #132 (timidri)
• change to http when client cert is false #131 (WBasson)
• Expose manage_repo in dbcompiler::install class #129 (suckatrash)

v2.4.0 (2020-12-30)

Full Changelog

Added

• (SUP-2007) Add local import and viewing of metrics #120 (jarretlavallee)
• Improve multiserver telegraf dashboard support #114 (seanmil)
• Add tidy_telegraf_configs option on dbcompiler profile #112 (seanmil)
• Use supplied compiler name in dbcompiler #111 (seanmil)
• Support multiple Postgres instances #110 (seanmil)

Fixed

• re-Add dashboard for JRuby per-borrow metrics #124 (suckatrash)
• Detect new cert_allowlist_entry class #119 (kreeuwijk)
• Fix raw queries in Postgres dashboard #117 (seanmil)
• Fix logic to select the v2 PuppetDB dashboards #113 (seanmil)

v2.3.1 (2020-08-11)

Full Changelog

Added

• Add dashboard for JRuby per-borrow metrics #106 (suckatrash)
• Replace httpjson with http input #105 (suckatrash)
• Make datasource for system metrics configurable #104 (m0dular)
• add description to InfluxDB yumrepo #98 (vchepkov)
• PuppetDB on Compilers #97 (coreymbe)
• Update PE PuppetDB metrics #95 (Sharpie)

Fixed

• Fix measurement names for Process dashboards #102 (m0dular)

UNCATEGORIZED PRS; LABEL THEM ON GITHUB

• (maint) v2.3.1 release prep #103 (suckatrash)

v2.3.0 (2020-04-15)

Full Changelog

Added

• Make the tidy resource optional #93 (suckatrash)

Fixed

• (DIO-563) addresses CVE-2020-7943 #92 (suckatrash)

v2.2.0 (2020-03-20)

Full Changelog

Added

• (SLV-788) New and updated dashboards #89 (RandellP)
• OSP Compatibility Updates #87 (jarretlavallee)
• Use the puppetserver.jruby metrics in the dashboard examples #79 (jarretlavallee)
• Added arbitrary grafana config #76 (dylanratcliffe)
• Added ability to set default password #75 (dylanratcliffe)

Fixed

• (maint) correct script name #90 (tkishel)
• Update certs.pp #73 (uberjew666)

v2.0.1 (2019-07-11)

Full Changelog

v2.0.0 (2019-07-11)

Full Changelog

Added

• Switching to in module hiera data & updating docs #65 (genebean)
• Improve PuppetDB metric dashboards #63 (Sharpie)
• Add profiles, move puppetdb metric defaults (part 2) #62 (suckatrash)
• Select PuppetDB metrics based on version #56 (seanmil)
• Update dashboard templates #52 (Sharpie)

Fixed

• Fix archive dashboard query for compile time #64 (Sharpie)
• Use certname consistently for key files #55 (seanmil)
• Ensure http timeout is set in all inputs.http* for telegraf #54 (jarretlavallee)

v1.1.5 (2019-04-25)

Full Changelog

Added

• Allow port numbers of services to be specified #50 (Sharpie)
• Improve support for FOSS puppet dashboards #49 (suckatrash)
• add puppetdb heap / status #39 (suckatrash)
• add a metric for last successful file-sync commit #36 (suckatrash)

Fixed

• Stop creating /run/grafana on CentOS 7 #38 (Sharpie)

1.1.0 (2019-02-13)

Full Changelog

Added

• Add postgresql metrics collection #30 (suckatrash)
• Add params, separate configs #20 (genebean)

1.0.3 (2018-12-01)

Full Changelog

Fixed

• Removed unless block from template #11 (genebean)

1.0.2 (2018-08-06)

Full Changelog

Fixed

• (maint) add license file #8 (suckatrash)

1.0.1 (2018-07-06)

Full Changelog

Added

• Merging up everything currently in development #4 (suckatrash)

Fixed

• (GH12) Update systemd tmpfiles #3 (jarretlavallee)

1.0.0 (2018-07-02)

Full Changelog

* This Changelog was automatically generated by github_changelog_generator