puppet_metrics_dashboard
Version information
This version is compatible with:
- Puppet Enterprise 2023.2.x, 2023.1.x, 2023.0.x, 2021.7.x, 2021.6.x, 2021.5.x, 2021.4.x, 2021.3.x, 2021.2.x, 2021.1.x, 2021.0.x, 2019.8.x, 2019.7.x, 2019.5.x, 2019.4.x, 2019.3.x, 2019.2.x, 2019.1.x, 2019.0.x, 2018.1.x, 2017.3.x, 2017.2.x, 2017.1.x, 2016.5.x, 2016.4.x
- Puppet >= 4.7.0 < 8.0.0
- , ,
This module has been deprecated by its author since Jul 14th 2022.
The author has suggested puppetlabs-puppet_operational_dashboards as its replacement.
Start using this module
Documentation
puppet_metrics_dashboard
Description
This module is used to configure Telegraf, InfluxDB, and Grafana, and collect, store, and display metrics collected from Puppet services. By default, those components are installed on a separate Dashboard node by applying the base class of this module to that node. That class will automatically query PuppetDB for Puppet Infrastructure nodes (Masters, PuppetDB hosts, PostgreSQL hosts) or you can specify them via associated class parameters. It is not recommended to apply the base class of this module to one of your Puppet Infrastructure nodes.
You have the option to use the included defined types to configure Telegraf to run on each Puppet Infrastructure node,
with the metrics being stored and displayed by another node running InfluxDB and Grafana.
In environments where there is an existing InfluxDB/Grafana installation, this option is recommended.
See Determining where Telegraf runs for details.
You have the option of collecting metrics using any or all of the following methods:
- Via Telegraf, which polls Puppet service endpoints (default, recommended)
- Via Puppet Server's built-in Graphite support (Section: Enabling Puppet Server's Graphite support)
- Via Archive files imported from the puppetlabs/puppet_metrics_collector module
Setup
In PuppetDB 6.9.1 & 5.2.13 and newer, the
/metrics/v1
endpoints are disabled by default and access to the/metrics/v2
endpoints are restricted to localhost only in response to CVE-2020-7943. Starting with version 2.3.0 of this module, PuppetDB metrics will not be setup by the main class if you are on the versions above or higher unless the main class is applied to the master. To collect PuppetDB metrics in other scenarios, you should use thepuppet_metrics_dashboard::profile::puppetdb
class applied to any PuppetDB nodes with the optionenable_client_cert => false
(the request will be to localhost and doen't require SSL)
Upgrade notes
- Version 2 and up now requires the
toml-rb
gem installed on the Master and any/all Compilers. - The
puppet_metrics_dashboard::profile::postgres
class is deprecated in favor of thepuppet_metrics_dashboard::profile::master::postgres_access
class. - Parameters
telegraf_agent_interval
andhttp_response_timeout
were previously Integers but are now Strings. The value should match a time interval, such as5s
,10m
, or1h
. influxdb_urls
was previously a String, but is now an Array.
Previous versions of this module added several [[inputs.httpjson]]
entries in /etc/telegraf/telegraf.conf
.
These entries should be removed, as all module-specific settings now reside in individual files within /etc/telegraf/telegraf.d/
.
Telegraf will continue to work if you do not remove them, however, the old [[inputs.httpjson]]
will not be updated going forward.
Determining where Telegraf runs
Telegraf can be configured to run on the Dashboard node, or on each Puppet Infrastructure node.
By default, this module configures Telegraf on the Dashboard node by querying PuppetDB to identify each Puppet Infrastructure node.
To manually configure Telegraf on the Dashboard node, define the following puppet_metrics_dashboard
class parameters: master_list
, puppetdb_list
and postgres_host_list
.
To configure Telegraf to run on each Puppet Infrastructure node, use the corresponding profiles for those nodes.
See Profile defined types.
Apply the puppet_metrics_dashboard
class to the Dashboard node to configure InfluxDB and Grafana, and apply the profile classes on each Puppet Infrastructure node to configure Telegraf.
Requirements
The toml-rb gem is a requirement of the puppet-telegraf
module, and needs to be installed in Puppet Server on the Master and any/all Compilers.
Apply the following class to the Master and any/all Compilers to install the gem.
node 'master.example.com' {
include puppet_metrics_dashboard::profile::master::install
}
node 'compiler.example.com' {
include puppet_metrics_dashboard::profile::master::install
}
Or, you can apply the puppet_metrics_dashboard::profile::master::install
class to the PE Master
Node Group, if using Puppet Enterprise.
Or, you can manually install the gem using the following command.
puppetserver gem install toml-rb
Restart the Puppet Server service after manually installing the gem.
If you are configuring the Dashboard node via a puppet apply
workflow, you will need to install the gem into Puppet on that host.
Usage
Configure a Monolithic Master and a Dashboard node
node 'master.example.com' {
include puppet_metrics_dashboard::profile::master::install
include puppet_metrics_dashboard::profile::master::postgres_access
}
node 'dashboard.example.com' {
class { 'puppet_metrics_dashboard':
add_dashboard_examples => true,
overwrite_dashboards => false,
}
}
This will configure Telegraf, InfluxDB, and Grafana on the Dashboard node, and allow Telegraf on that host to access PostgreSQL on the Monolithic Master.
Note that the add_dashboard_examples
parameter enforces state on the example dashboards.
Setting the overwrite_dashboards
parameter to true
disables overwriting your modifications (if any) to the example dashboards.
Manual configuration of a complex Puppet Infrastructure
node 'master.example.com' {
include puppet_metrics_dashboard::profile::master::install
}
node 'compiler01.example.com' {
include puppet_metrics_dashboard::profile::master::install
}
node 'compiler02.example.com' {
include puppet_metrics_dashboard::profile::master::install
}
node 'postgres01.example.com' {
include puppet_metrics_dashboard::profile::master::postgres_access
}
node 'postgres02.example.com' {
include puppet_metrics_dashboard::profile::master::postgres_access
}
node 'dashboard.example.com' {
class { 'puppet_metrics_dashboard':
add_dashboard_examples => true,
overwrite_dashboards => false,
configure_telegraf => true,
enable_telegraf => true,
master_list => ['master.example.com', ['compiler01.example.com', 9140], ['compiler02.example.com', 9140]],
puppetdb_list => ['puppetdb01.example.com', 'puppetdb02.example.com'],
postgres_host_list => ['postgres01.example.com', 'postgres02.example.com'],
}
}
# Alternate ports are configured using a pair of: [host_name, port_number]
Note that the defaults for this module's class parameters are defined in its data/common.yaml
directory.
The *_list
parameters can be defined in the class declaration, or elsewhere in Hiera. For example:
puppet_metrics_dashboard::master_list:
- "master.example.com"
- ["compiler01.example.com", 9140]
- ["compiler02.example.com", 9140]
puppet_metrics_dashboard::puppetdb_list:
- "puppetdb01.example.com"
- "puppetdb02.example.com"
puppet_metrics_dashboard::postgres_host_list:
- "postgres01.example.com"
- "postgres02.example.com"
Configure Master, Compiler running PuppetDB and a Dashboard node
node 'master.example.com' {
include puppet_metrics_dashboard::profile::master::install
}
node 'dbcompiler.example.com' {
class { 'puppet_metrics_dashboard::profile::dbcompiler::install':
influxdb_urls => ["http://dashboard.example.com:8086"]
}
}
node 'dashboard.example.com' {
class { 'puppet_metrics_dashboard':
add_dashboard_examples => true,
overwrite_dashboards => false,
}
}
This will configure Telegraf, InfluxDB, and Grafana on the Dashboard node, and allow Telegraf on that host to access PuppetDB on the Compiler running PuppetDB.
Configure Graphite
node 'dashboard.example.com' {
class { 'puppet_metrics_dashboard':
add_dashboard_examples => true,
overwrite_dashboards => false,
consume_graphite => true,
influxdb_database_name => ['graphite'],
master_list => ['master', 'master02'],
}
}
- This method requires enabling Graphite on the Masters, as described here.
The hostnames that you use in
master_list
must match the value(s) that you used formetrics_server_id
in thepuppet_enterprise::profile::master
class. You must use hostnames rather than fully-qualified domain names (no dots) both in this class and in thepuppet_enterprise::profile::master
class.
Configure Telegraf, Graphite, and Archive
Archive refers to files imported from the puppetlabs/puppet_metrics_collector module.
node 'dashboard.example.com' {
class { 'puppet_metrics_dashboard':
add_dashboard_examples => true,
overwrite_dashboards => false,
configure_telegraf => true,
enable_telegraf => true,
consume_graphite => true,
influxdb_database_name => ['telegraf', 'graphite', 'puppet_metrics'],
}
}
Import Archive Metrics
The json2timeseriesdb
script from the puppetlabs/puppet_metrics_collector module can be used to transform its data and import it into InfluxDB.
Examples:
./json2timeseriesdb /opt/puppetlabs/puppet-metrics-collector/puppetserver/*/*.json --convert-to influxdb --influx-db puppet_metrics --netcat dashboard.example.com
This simple example can be used for small number of files. For a large number of files, use --pattern
.
./json2timeseriesdb --pattern '/opt/puppetlabs/puppet-metrics-collector/puppetserver/*/*.json' --convert-to influxdb --influx-db puppet_metrics --netcat dashboard.example.com
The --pattern
flag accepts a Ruby glob argument, which the script will internally expand into a list of files.
Allow Telegraf to access PE-PostgreSQL
The following class is required to be applied to the Master (or the PE Database node if using external PostgreSQL) for collection of PostgreSQL metrics via Telegraf.
node 'master.example.com' {
class { 'puppet_metrics_dashboard::profile::master::postgres_access':
telegraf_host => 'grafana-server.example.com',
}
}
The telegraf_host
parameter is optional.
By default, the class will query PuppetDB for Dashboard nodes (with the puppet_metrics_dashboard
class applied) and use the certname
of the first node in the results.
If the PuppetDB lookup fails to find a Dashboard node, and you do not specify telegraf_host
then the class outputs a warning.
Refer to Issue 72 if the above generates the following error:
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Function Call, 'versioncmp' parameter 'a' expects a String value, got Undef (file: /opt/puppetlabs/puppet/modules/pe_postgresql/manifests/server/role.pp, line: 66, column: 6) (file: /etc/puppetlabs/code/environments/production/modules/puppet_metrics_dashboard/manifests/profile/master/postgres_access.pp, line: 42) on node master.example.com
A workaround for that error is to apply the puppet_metrics_dashboard::profile::master::postgres_access
class to the PE Database
Node Group in the Console, if using Puppet Enterprise.
Enable SSL
node 'dashboard.example.com' {
class { 'puppet_metrics_dashboard':
use_dashboard_ssl => true,
}
}
By default, this will create a set of certificates in /etc/grafana
that are based on the Dashboard node's Puppet agent certificates.
You can also specify different files by defining the dashboard_cert_file
and dashboard_cert_key
parameters, but managing certificate content or supplying your own certificates is unsupported by this module.
Note that enabling SSL on Grafana will not allow for running on privileged ports such as 443
.
To enable that capability, use the suggestions documented in the Grafana documentation
Profile defined types
The module includes defined types that you can use with an existing Grafana implementation. See REFERENCE.md for example usage.
Note that because of the way that the Telegraf module works, these examples will overwrite any configuration in telegraf.conf
if it is not already managed by Puppet.
See the puppet-telegraf documentation on how to manage that file and add other settings.
Other possibilities
Configure the password for InfluxDB , enable additional TICK Stack components, and customize Grafana.
node 'dashboard.example.com' {
class { 'puppet_metrics_dashboard':
influx_db_password => 'secret',
enable_chronograf => true,
enable_kapacitor => true,
grafana_http_port => 3333,
grafana_version => '6.5.2',
}
}
Reference
This module is documented via pdk bundle exec puppet strings generate --format markdown
.
Please refer to REFERENCE.md for more information.
Limitations
Repository failure for InfluxDB packages
When installing InfluxDB on CentOS/RedHat 6/7 you may encounter the following error message.
Error: Execution of '/usr/bin/yum -d 0 -e 0 -y install telegraf' returned 1: Error: Cannot retrieve repository metadata (repomd.xml) for repository: influxdb. Please verify its path and try again
Error: /Stage[main]/Puppet_metrics_dashboard::Telegraf/Package[telegraf]/ensure: change from purged to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y install telegraf' returned 1: Error: Cannot retrieve repository metadata (repomd.xml) for repository: influxdb. Please verify its path and try again
This is due to a mismatch in the ciphers available in the operating system and on the InfluxDB repository.
To resolve this issue, update nss
and curl
on the Dashboard node.
yum install curl nss --disablerepo influxdb
PostgreSQL metrics collection with older versions of Telegraf
PostgreSQL metrics collection requires Telegraf version 1.9.1 or later.
Development
Please refer to CONTRIBUTING.md for more information.
Reference
Table of Contents
Classes
Public Classes
puppet_metrics_dashboard
: Installs and configures a stack for collecting, storing, and displaying Puppet Infrastructure metricspuppet_metrics_dashboard::profile::dbcompiler::install
: Apply this class to compilers running PuppetDB to configure Telegraf and collect puppetserver and puppetdb metricspuppet_metrics_dashboard::profile::master::install
: Install requirements for the voxpupuli/puppet-telegraf modulepuppet_metrics_dashboard::profile::master::postgres_access
: Apply this class to a PE PostgreSQL node to allow access by Telegraf.puppet_metrics_dashboard::profile::postgres
: This class is deprecated. Please use the Puppet_metrics_dashboard::Profile::Master::Postgres_access class.
Private Classes
puppet_metrics_dashboard::config
: Configures dashboard components.puppet_metrics_dashboard::dashboards
: Configures Grafana dashboards.puppet_metrics_dashboard::dashboards::graphite
: Installs graphite example dashboardspuppet_metrics_dashboard::dashboards::puppet_metrics
: Installs puppet_metrics example dashboardspuppet_metrics_dashboard::dashboards::telegraf
: Installs telegraf example dashboardspuppet_metrics_dashboard::grafana
: Install and configure Grafanapuppet_metrics_dashboard::install
: Installs InfluxDB components.puppet_metrics_dashboard::post_start_configs
: InfluxDB post-start configspuppet_metrics_dashboard::repos
: Configures InfluxDB and Grafana repospuppet_metrics_dashboard::service
: Manages servicespuppet_metrics_dashboard::telegraf
: Install and configure Telegrafpuppet_metrics_dashboard::telegraf::config
: Configures Telegrafpuppet_metrics_dashboard::telegraf::service
: Manages the Telegraf service
Defined types
puppet_metrics_dashboard::certs
: This class creates a certificates for Grafana and for connecting to PE Postgres.puppet_metrics_dashboard::profile::compiler
: Apply this class to a master or compiler to collect puppetserver metricspuppet_metrics_dashboard::profile::master::postgres
: Apply this class to an agent running pe-postgresql to collect postgres metricspuppet_metrics_dashboard::profile::puppetdb
: Apply this class to a node running puppetdb to collect puppetdb metrics
Functions
puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile
puppet_metrics_dashboard::puppetdb_metrics
: function used to determine the set of needed PuppetDB metrics based on PE versionpuppet_metrics_dashboard::puppetdb_no_remote_metrics
Data types
Puppet_metrics_dashboard::HostList
: A list of hostnames, or pairs of hostname and port.Puppet_metrics_dashboard::Puppetdb_metric
: A metric name corresponding to an endpoint (url)
Classes
puppet_metrics_dashboard
Installs and configures a stack for collecting, storing, and displaying Puppet Infrastructure metrics.
Refer to data/common.yaml
for additional parameter defaults.
Examples
Grafana with no login
class { 'puppet_metrics_dashboard':
grafana_config => {
'users' => {
'allow_sign_up' => false,
},
'auth.anonymous' => {
'enabled' => true,
},
},
}
Configure Telegraf to collect metrics from a list of Masters, PuppetDB, and PostgreSQL servers
class { 'puppet_metrics_dashboard':
add_dashboard_examples => true,
overwrite_dashboards => false,
configure_telegraf => true,
enable_telegraf => true,
master_list => ['master.example.com', ['compiler01.example.com', 9140], ['compiler02.example.com', 9140]],
puppetdb_list => ['puppetdb01.example.com', 'puppetdb02.example.com'],
postgres_host_list => ['postgres01.example.com', 'postgres02.example.com'],
}
Configure Graphite to accept metrics from a list of Masters
class { 'puppet_metrics_dashboard':
add_dashboard_examples => true,
overwrite_dashboards => false,
consume_graphite => true,
influxdb_database_name => ['graphite'],
master_list => ['master', 'master02'],
}
Configure Telegraf, Graphite, and Archive
class { 'puppet_metrics_dashboard':
add_dashboard_examples => true,
overwrite_dashboards => false,
consume_graphite => true,
configure_telegraf => true,
enable_telegraf => true,
influxdb_database_name => ['telegraf', 'graphite', 'puppet_metrics'],
}
Parameters
The following parameters are available in the puppet_metrics_dashboard
class.
manage_repos
Data type: Boolean
Whether to configure apt / yum repositories for required packages.
add_dashboard_examples
Data type: Boolean
Whether to add the example Grafana dashboards for the configured InfluxDB databases. Defaults to false
.
Note: These dashboards are managed and any changes will be overwritten unless the overwrite_dashboards
is set to false
.
overwrite_dashboards
Data type: Boolean
Whether to overwrite the example Grafana dashboards. Defaults to true
This parameter disables overwriting the example Grafana dashboards.
It takes effect after the second Puppet run, and populates a overwrite_dashboards_disabled
fact.
Only used when add_dashboard_examples
is true
.
enable_chronograf
Data type: Boolean
Whether to install chronograf. Defaults to false
No configuration of chronograf is included at this time.
enable_kapacitor
Data type: Boolean
Whether to install kapacitor. Defaults to false
No configuration of kapacitor is included at this time.
enable_telegraf
Data type: Boolean
Whether to install telegraf. Defaults to true
No configuration is done unless configure_telegraf
is set to true
.
configure_telegraf
Data type: Boolean
Whether to configure the Telegraf service. Defaults to true
This parameter enables and configures Telegraf to query the *_list
hosts for metrics.
Metrics will be stored in the telegraf
database in InfluxDb.
Ensure that influxdb_database_name
contains telegraf
when using this parameter.
Only used when enable_telegraf
is true
.
consume_graphite
Data type: Boolean
Whether to enable the InfluxDB Graphite plugin. Defaults to false
This parameter enables the Graphite plugin for InfluxDB to allow for consuming Graphite metrics.
Ensure influxdb_database_name
contains graphite
when using this parameter.
Note: To consume metrics sent from Puppet Server, this must to be set to true
.
influxdb_database_name
Data type: Array[String]
An Array of databases that should be created in InfluxDB.
Valid values are telegraf
, graphite
, puppet_metrics
, and any other string. Defaults to ["telegraf"]
Each database in the array will be created in InfluxDB.
telegraf
, graphite
, and puppet_metrics
are specially named and will be used with their associated metric collection method.
Any other database name will be created, but not associated with components in this module.
influxdb_urls
Data type: Array[String]
An Array containing urls defining InfluxDB instances for Telegraf.
influx_db_service_name
Data type: String
Name of the InfluxDB service used by the operating system.
influx_db_password
Data type: String
The password for the InfluxDB admin
user.
Defaults to puppet
telegraf_db_name
Data type: String
The InfluxDB database where Telegraf metrics are stored.
http_response_timeout
Data type: String[2]
Timeout for Telegraf HTTP requests. Defaults to 5s
telegraf_agent_interval
Data type: String[2]
Frequency of Telegraf HTTP queries for metrics. Defaults to 5s
pg_query_interval
Data type: String[2]
Frequency of Telegraf PostgreSQL queries for metrics. Defaults to 10m
use_dashboard_ssl
Data type: Boolean
Whether to enable SSL in Grafana.
Valid values are true
, false
. Defaults to false
tidy_telegraf_configs
Data type: Boolean
Whether or not to remove unmanaged configuration files from /etc/telegraf/telegraf.d
. Defaults to false
dashboard_cert_file
Data type: String
The location of the Grafana certficiate.
Defaults to /etc/grafana/${clientcert}_cert.pem
Only used when use_dashboard_ssl
is true
.
dashboard_cert_key
Data type: String
The location of the Grafana private key.
Defaults to /etc/grafana/${clientcert}_key.pem
Only used when use_dashboard_ssl
is true
.
grafana_http_port
Data type: Integer
The port for the Grafana web interface.
Valid values are Integers from 1024
to 65536
. Defaults to 3000
This should be a nonprivileged port (above 1024).
grafana_password
Data type: String
The password for the Grafana admin
user.
Defaults to admin
grafana_version
Data type: String
The version of Grafana to install. Valid values are String versions of Grafana.
overwrite_dashboards_file
Data type: String
File used to populate the overwrite_dashboards
fact.
grafana_config
Data type: Hash
Hash of arbitrary configuration settings to pass to Grafana.
These are added to grafana.ini
with top-level keys becoming sections and their key-value children becoming settings.
master_list
Data type: Puppet_metrics_dashboard::HostList
An Array of servers that Telegraf will collect Puppet Server metrics from. Entries may be:
- A String that contains a hostname or IP address.
(The module will use a default port number of
8140
) - An Array where the first entry is a String that contains a hostname or IP address,
and the second entry is an Integer that specifies the port number.
Defaults to the result of a PuppetDB query, or
[$trusted['certname']]
Default value: puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile('master')
puppetdb_list
Data type: Puppet_metrics_dashboard::HostList
An Array of servers that Telegraf will collect PuppetDB metrics from. Entries may be:
- A String that contains a hostname or IP address.
(The module will use a default port number of
8081
) - An Array where the first entry is a String that contains a hostname or IP address,
and the second entry is an Integer that specifies the port number.
Defaults to the result of a PuppetDB query, or
[$trusted['certname']]
Default value: puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile('puppetdb')
postgres_host_list
Data type: Puppet_metrics_dashboard::HostList
An Array of servers that Telegraf will collect PostgreSQL metrics from. Entries may be:
- A String that contains a hostname or IP address.
(The module will use a default port number of
5432
) - An Array where the first entry is a String that contains a hostname or IP address,
and the second entry is an Integer that specifies the port number.
Defaults to the results of a PuppetDB query, or
[$trusted['certname']]
Default value: puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile('database')
puppetdb_metrics
Data type: Puppet_metrics_dashboard::Puppetdb_metric
An Array of Hashes containing name/url pairs for each PuppetDB metric.
Refer to functions/puppetdb_metrics.pp
for defaults.
Default value: puppet_metrics_dashboard::puppetdb_metrics()
puppet_metrics_dashboard::profile::dbcompiler::install
Apply this class to compilers running PuppetDB to configure Telegraf and collect puppetserver and puppetdb metrics
Parameters
The following parameters are available in the puppet_metrics_dashboard::profile::dbcompiler::install
class.
influxdb_urls
Data type: Array[String]
An Array containing urls defining InfluxDB instances for Telegraf.
timeout
Data type: String[2]
Default timeout of http calls. Defaults to 5 seconds
Default value: lookup('puppet_metrics_dashboard::http_response_timeout')
compiler
Data type: Variant[String,Tuple[String, Integer]]
The FQDN of the compiler / master. Defaults to the FQDN of the server where the profile is applied
Default value: $facts['networking']['fqdn']
puppetdb_host
Data type: Variant[String,Tuple[String, Integer]]
Where to query the puppetdb host. Defaults to localhost.
Default value: 'localhost'
cm_port
Data type: Integer[1]
The port that the puppetserver service listens on on your compiler. Defaults to 8140
Default value: 8140
db_port
Data type: Integer[1]
The port that the puppetdb service listens on on your compiler. Defaults to 8081
Default value: 8081
interval
Data type: String[2]
The frequency that telegraf will poll puppetserver metrics. Defaults to '5s'
Default value: '5s'
tidy_telegraf_configs
Data type: Boolean
Whether or not to remove unmanaged configuration files from /etc/telegraf/telegraf.d
. Defaults to false
.
Default value: lookup('puppet_metrics_dashboard::tidy_telegraf_configs')
puppetdb_metrics
Data type: Puppet_metrics_dashboard::Puppetdb_metric
Default value: puppet_metrics_dashboard::puppetdb_metrics()
puppet_metrics_dashboard::profile::master::install
Install requirements for the voxpupuli/puppet-telegraf module.
Examples
Apply this class to the Master and any/all Compilers
include puppet_metrics_dashboard::profile::master::install
puppet_metrics_dashboard::profile::master::postgres_access
Apply this class to a PE PostgreSQL node to allow access by Telegraf.
Examples
Apply this class to PE PostgreSQL nodes
class { 'puppet_metrics_dashboard::profile::master::postgres_access':
telegraf_host => 'dashboard.example.com',
}
Parameters
The following parameters are available in the puppet_metrics_dashboard::profile::master::postgres_access
class.
telegraf_host
Data type: Optional[String[1]]
The FQDN of the host running Telegraf. Defaults to an empty string. You can define this parameter, otherwise this class will query PuppetDB for a dashboard host.
Default value: undef
puppet_metrics_dashboard::profile::postgres
This class is deprecated. Please use the Puppet_metrics_dashboard::Profile::Master::Postgres_access class.
Parameters
The following parameters are available in the puppet_metrics_dashboard::profile::postgres
class.
grafana_host
Data type: String
The FQDN of the host where telegraf runs. Defaults to an empty string. You can explicitly set this parameter or the class attempts to lookup which host has the puppet_metrics_dashboard class applied in PuppetDB. If the parameter is not set and the lookup does not return anything we issue a warning.
Default value: ''
Defined types
puppet_metrics_dashboard::certs
This class creates a set of certificates in /etc/${service}. These certificates are used when configuring Grafana to use SSL and to connect to PE Postgres. The certificates are based on the agent's own Puppet certificates.
Parameters
The following parameters are available in the puppet_metrics_dashboard::certs
defined type.
service
Data type: Any
The service name associated with these certificates.
Default value: $name
puppet_metrics_dashboard::profile::compiler
Apply this class to a master or compiler to collect puppetserver metrics
Examples
Add telegraf to a master / compiler
puppet_metrics_dashboard::profile::compiler{ $facts['networking']['fqdn']:
timeout => '5s',
}
Parameters
The following parameters are available in the puppet_metrics_dashboard::profile::compiler
defined type.
timeout
Data type: String[2]
Default timeout of http calls. Defaults to 5 seconds
Default value: lookup('puppet_metrics_dashboard::http_response_timeout')
compiler
Data type: Variant[String,Tuple[String, Integer]]
The FQDN of the compiler / master. Defaults to the FQDN of the server where the profile is applied
Default value: $facts['networking']['fqdn']
port
Data type: Integer[1]
The port that the puppetserver service listens on on your compiler. Defaults to 8140
Default value: 8140
interval
Data type: String[2]
The frequency that telegraf will poll for metrics. Defaults to '5s'
Default value: '5s'
puppet_metrics_dashboard::profile::master::postgres
Apply this class to an agent running pe-postgresql to collect postgres metrics
Examples
Add telegraf to a postgres server
puppet_metrics_dashboard::profile::master::postgres{ $facts['networking']['fqdn']:
query_interval => '10m',
}
Parameters
The following parameters are available in the puppet_metrics_dashboard::profile::master::postgres
defined type.
query_interval
Data type: String[2]
How often to run the queries in minutes. Defaults to 10 minutes.
Default value: lookup('puppet_metrics_dashboard::pg_query_interval')
postgres_host
Data type: Variant[String,Tuple[String, Integer]]
The FQDN of the postgres host. Defaults to the FQDN of the server where the profile is applied
Default value: $facts['networking']['fqdn']
port
Data type: Integer[1]
The port that the postgres service listens on. Defaults to 5432
Default value: 5432
databases
Data type: Array[String[1]]
An Array of databases to query on. Defaults to ['pe-puppetdb','pe-rbac','pe-activity','pe-classifier']
Default value: ['pe-puppetdb','pe-rbac','pe-activity','pe-classifier']
puppet_metrics_dashboard::profile::puppetdb
Apply this class to a node running puppetdb to collect puppetdb metrics
Examples
Add telegraf to a puppetdb node
puppet_metrics_dashboard::profile::puppetdb{ $facts['networking']['fqdn']:
timeout => '5s',
puppetdb_metrics => puppet_metrics_dashboard::puppetdb_metrics(), # this is the default value
}
Parameters
The following parameters are available in the puppet_metrics_dashboard::profile::puppetdb
defined type.
timeout
Data type: String[2]
Default timeout of http calls. Defaults to 5 seconds
Default value: lookup('puppet_metrics_dashboard::http_response_timeout')
puppetdb_metrics
Data type: Puppet_metrics_dashboard::Puppetdb_metric
An array of hashes containing name / url pairs for each puppetdb metric. See functions/puppetdb_metrics.pp for defaults.
Default value: puppet_metrics_dashboard::puppetdb_metrics()
puppetdb_host
Data type: Variant[String,Tuple[String, Integer]]
Where to query the puppetdb host. Defaults to localhost.
Default value: 'localhost'
port
Data type: Integer[1]
The port that the puppetdb service listens on on your compiler. Defaults to 8081
Default value: 8081
interval
Data type: String[2]
The frequency that telegraf will poll for metrics. Defaults to '5s'
Default value: '5s'
enable_client_cert
Data type: Boolean
A boolean to enable using the client certificate for the PuppetDB queries. Defaults to true
Default value: true
Functions
puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile
Type: Puppet Language
The puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile function.
puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile(String $profile)
The puppet_metrics_dashboard::localhost_or_hosts_with_pe_profile function.
Returns: Array
profile
Data type: String
puppet_metrics_dashboard::puppetdb_metrics
Type: Puppet Language
The list of metrics to pull from PuppetDB depends on the PE version. To avoid having a data file for each version we utilize this function to build the needed array of hashes.
puppet_metrics_dashboard::puppetdb_metrics()
The list of metrics to pull from PuppetDB depends on the PE version. To avoid having a data file for each version we utilize this function to build the needed array of hashes.
Returns: Array[Hash]
An array of hashes containing name / url pairs for each puppetdb metric.
puppet_metrics_dashboard::puppetdb_no_remote_metrics
Type: Puppet Language
The puppet_metrics_dashboard::puppetdb_no_remote_metrics function.
puppet_metrics_dashboard::puppetdb_no_remote_metrics()
The puppet_metrics_dashboard::puppetdb_no_remote_metrics function.
Returns: Boolean
Data types
Puppet_metrics_dashboard::HostList
A list of hostnames, or pairs of hostname and port.
Alias of Array[Variant[ String, Tuple[String, Integer] ]]
Puppet_metrics_dashboard::Puppetdb_metric
A metric name corresponding to an endpoint (url)
Alias of Tuple[Struct[{ name => String[1], url => String[1] }], 1, default]
Change log
All notable changes to this project will be documented in this file. The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
2020-12-30 - Release - 2.4.0
Changes
- Adds puppet7 testing / support
- Support for multiple Postgres instances
- Use supplied compiler name in dbcompiler profile
- Add tidy_telegraf_configs option on dbcompiler profile
- Introduces new cert_allowlist_entry class
- Add local import and viewing of metrics
- Updates the contrib docs for litmus
- Re-adds dashboard for JRuby per-borrow metrics
Bugfixes
- Fix logic to select the v2 PuppetDB dashboards
- Improve multiserver telegraf dashboard support
- Fix raw queries in Postgres dashboard
2020-7-24 - Release - 2.3.1
Changes
- Added the
puppet_metrics_dashboard::profile::dbcompiler::install
class to mitigate CVE-2020-7943. - Added a description to the influxdb yumrepo
- Updated Puppet VS Code Extension ID
- Datasource is now configurable on te system metrics graphs
- Replaced deprecated httpjson telegraf input with http
- Added a dashboard for Telegraf Puppetserver workload
Bugfixes
- Fixed measurement names for process dashboards
- CI fixes
2020-4-15 - Release - 2.3.0
Changes
- Account for CVE-2020-7943 by configuring telegraf to collect PuppetDB metrics from localhost only on the v2 metrics endpoint, updates dashboards to accomodate new values.
- Added a tidy resource that cleans up unmanaged telegraf configuration in
/etc/telegraf/telegraf.d
iftidy_telegraf_configs
is set to true in the main class (default is false) - Converted the acceptance tests from beaker to litmus
Bugfixes
2020-3-20 - Release - 2.2.0
Changes
- New and updated dashboards
- Updated documentation
- Use the client SSL certificate when querying PuppetDB metrics with Telegraf
- Allow for specifying the PostgreSQL databases in the PostgreSQL profile
- The certs defined type now uses a
file://
source since it's copying locally - Added the ability to set the Grafana admin password on initial deployment
- Added the ability to pass arbitrary config options to Granfana
Bugfixes
- Remove the file sync query from PuppetDB metrics when
pe_server_version
is not defined when using Telegraf - Update the included dashboards to not reference deprecated
pe-
metrics - Compatibility fixes with OSP, including dashboards
- Update module dependencies
2019-7-11 - Release - 2.0.1
Bugfixes
- Document a requirement on the toml-rb rem
2019-7-11 - Release - 2.0.0
Changes
- The module now supports configuring Telegraf on each Puppet infrastructure node in addition to configuring it on the same server that runs Grafana. This is to better support end-users that already have a Grafana instance and just want to collect Puppet metrics. A set of profile classes has been added to allow for this.
- Improvements to dashboard templates for Telegraf and archive.
- Cleanup PuppetDB metrics in some versions.
$telegraf_agent_interval
and$http_response_timeout
in the main class are now Strings (previously integers) they should looks like: '5s', '2m' or '1h'.$influxdb_urls
has changed from a String to an Array. This is to support multiple Influxdb backends.- Instead of defining all of Telegraf's metrics in a single file
/etc/telegraf/telegraf.d/puppet_metrics_dashboard.conf
, there will now be multiple files for each metric. An additional resource ensures that the old file is absent. - Replaced
params.pp
with module level hiera data. - Created a function for determining the needed PuppetDB metrics based on PE version.
- Removed the variable 'storage_metrics_db_queries' since it was not referenced anywhere.
Bugfixes
- Fix for Telegraf http timeouts not being set
- Fix for cases where certname != FQDN
2019-4-25 - Release - 1.1.5
Changes
- Added a metric for last successful file-sync commit
- Added puppetdb heap / status metrics
- Improve FOSS puppet support
- Allow port numbers of services to be specified
Bugfixes
- Stop creating /run/grafana on CentOS 7
- Cleanup /run/grafana spec tests for ubuntu
- Fixed grafana version req
2019-2-13 - Release - 1.1.0
Changes
- Code refactor to more standard layout
- Various CI and testing updates
- Updated apt-get / yum repo resources
- New feature: postgres metrics
- Moved telegraf config file from
/etc/telegraf/telegraf.conf
to/etc/telegraf/telegraf.d/puppet_metrics_dashboard.conf
- The SSL dashboard option no longer relies on puppetlabs/puppet_agent
- Tested and working on PE 2019.0.x
2018-11-30 - Release - 1.0.3
Changes:
- Minor fix for dependency versions
2018-08-03 - Release - 1.0.2
Changes:
- Added the missing license file.
2018-07-06 - Release - 1.0.1
Bugfixes
- Fixed an issue with RHEL7 where the grafana service wouldn't start after rebooting
- Fixed an issue with metadata.json where some of the URLs were incorrect
2018-07-02 - Release - 1.0.0
Initial forge release
Dependencies
- puppet-grafana (>= 3.0.0 < 8.0.0)
- puppet-telegraf (>= 2.0.0 < 4.0.0)
- puppetlabs-apt (>= 4.3.0 < 8.0.0)
- puppetlabs-inifile (>= 2.0.0 < 5.0.0)
- puppetlabs-puppetserver_gem (>= 1.1.1 < 3.0.0)
- puppetlabs-stdlib (>= 1.0.0 < 7.0.0)
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "{}" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright 2018 Erik Hansen Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.