Forge Home

cfmetrics

Centralized metrics collection and monitoring solution.

7,154 downloads

3,332 latest version

5.0 quality score

We run a couple of automated
scans to help you access a
module's quality. Each module is
given a score based on how well
the author has formatted their
code and documentation and
modules are also checked for
malware using VirusTotal.

Please note, the information below
is for guidance only and neither of
these methods should be considered
an endorsement by Puppet.

Version information

  • 1.3.1 (latest)
  • 1.3.0
  • 1.1.0
  • 1.0.1
  • 1.0.0
released Jun 16th 2019
This version is compatible with:
  • Puppet Enterprise 2019.8.x, 2019.7.x, 2019.5.x, 2019.4.x, 2019.3.x, 2019.2.x, 2019.1.x, 2019.0.x, 2018.1.x, 2017.3.x, 2017.2.x, 2017.1.x, 2016.5.x, 2016.4.x
  • Puppet >=4.7.0 <7.0.0
  • ,

Start using this module

  • r10k or Code Manager
  • Bolt
  • Manual installation
  • Direct download

Add this module to your Puppetfile:

mod 'codingfuture-cfmetrics', '1.3.1'
Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add codingfuture-cfmetrics
Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install codingfuture-cfmetrics --version 1.3.1

Direct download is not typically how you would use a Puppet module to manage your infrastructure, but you may want to download the module in order to inspect the code.

Download

Documentation

codingfuture/cfmetrics — version 1.3.1 Jun 16th 2019

cfmetrics

Centralized metrics collection and monitoring solution.

Description

  • Neutral collector interface:
    • netdata (default)
      • Real-time monitoring
      • Easy integration with Graphite, OpenTSDB and Prometheus
      • Very fast and resource efficient
      • Easily extendable
      • StatsD emulation
      • Integrated with cfsystem memory distribution.
  • Time series database:
    • prometheus
      • Scraps directly from nedata
  • Alerts:
    • Simple Alerta for plain Netdata case
    • AlertManager for Prometheus case
  • Specialized alert managing software support
    • fine control of notification methods
    • advanced filtering & history
    • access control
  • Plug & Play integration with other cf* modules

Terminology & Concept

  • Collector - abstract definition of collecting feature.
  • cfsystem::metric - abstract declaration of "collectable" resource.
  • Target - host to send metrics to collector-defined way.
  • Alert - preconfigured notification for infrastructure health state.
  • AlertManager - special high-available solution for Alert notifications like Alerta.io

Collector gather all system info it can. All cf* modules declare support for metrics of various services. If cfmetrics module is loaded then the declarations are used to automatically discover resources to monitor.

Netdata collector

Netdata allows building efficient data gathering topology. By default, all instances act in standalone mode with own history. If collector target is configured, then netdata does not maintain history - only push buffer.

It's possible to run own netdata registry.

Both binary and source installations are possible. Automatic update is tried on Puppet catalog run, but not more often than once in 1 hour.

Just for reference, a special LogStash instance accepting TSDB input format is supported to store metrics in Elasticsearch the efficient way. It suits small scale to unify logging and metrics centralization. For larger cases, Prometheus is suggested.

Even with configured AlertManager, critical Alerts are duplicated via email.

Technical Support

Setup

Up to date installation instructions are available in Puppet Forge: https://forge.puppet.com/codingfuture/cfmetrics

Please use librarian-puppet or cfpuppetserver module to deal with dependencies.

There is a known r10k issue RK-3 which prevents automatic dependencies of dependencies installation.

API

cfmetrics class

Main class of the module.

  • $collect = true - enabled collector
  • $iface = 'local' - default iface to bind services to

cfmetrics::collector class

Generic collector functionality.

  • $type = 'netdata' - collector implementation to use

cfmetrics::netdata class

  • Standard cfsystem resource limits:
    • $memory_weight = 1
    • $memory_min = undef
    • $memory_max = undef
    • $cpu_weight = 10
    • $io_weight = 10
  • $iface = $cfmetrics::iface - interface to listed for requests
  • $port = undef - networks port to bind
  • $target = undef - configure upstream target (hostname)
  • $server = false - act as server (upstream target)
  • $registry = false - enable local netdata registry
  • $registry_url = undef - setup non-default registry URL
  • $settings_tune = {} - fine tune of netdata configuration
    • all keys directly go to YAML, except special cfmetrics:
      • base_mem = 48 - base memory for netdata (requires noticeable amount for Python)
      • history_mem = 10 - how much memory to reserve per single host history
  • $binary_install = true - use binary install instead of git source based
  • $mirror = 'https://my-netdata.io' - what mirror to use for install script (system proxy aware)
  • $extra_clients = [] - define static list of possible netdata clients
  • $logstash = undef - LogStash TSDB-mode configuration support
  • $alerta = undef - define Alerta.io API endpoint support:
    • url - endpoint URL
    • key - secret API key
    • env - Alerta.io environment (scope)
  • $alarm_conf = {} - fine tune of alarm config

cfmetrics::prometheus class

Uses cfweb to run up to date prom/prometheus:latest container.

  • $memory_weight = 1

  • $memory_min = 256

  • $memory_max = undef

  • $cpu_weight = 10

  • $io_weight = 10

  • $iface = $cfmetrics::iface

  • $port = 9090

  • $prometheus_tune = {}

  • $image =

    • image => 'prom/prometheus'
    • image_tag => 'latest'
  • $server_name = "prometheus.${::facts['fqdn']}"

  • $site_params = {}

  • $rules = {}

cfmetrics::alertmanager class

Configures AlertManager for Prometheus