Forge Home

Modules

Puppet

Resources

PuppetEcosystemPuppet CommunityUpdates

puppetdbquery

Query functions for the PuppetDB API
Project URLRSS FeedReport issues

1,015,383 downloads

128,042 latest version

4.1 quality score

We run a couple of automated
scans to help you access a
module's quality. Each module is
given a score based on how well
the author has formatted their
code and documentation and
modules are also checked for
malware using VirusTotal.

Please note, the information below
is for guidance only and neither of
these methods should be considered
an endorsement by Puppet.

Support the Puppet Community by contributing to this module

You are welcome to contribute to this module by suggesting new features, currency updates, or fixes. Every contribution is valuable to help ensure that the module remains compatible with the latest Puppet versions and continues to meet community needs. Complete the following steps:

  1. Review the module’s contribution guidelines and any licenses. Ensure that your planned contribution aligns with the author’s standards and any legal requirements.
  2. Fork the repository on GitHub, make changes on a branch of your fork, and submit a pull request. The pull request must clearly document your proposed change.

For questions about updating the module, contact the module’s author.

Version information

  • 3.0.1 (latest)
  • 3.0.0
  • 2.3.0
  • 2.2.0
  • 2.1.1
  • 2.1.0
  • 2.0.3
  • 2.0.2
  • 2.0.1
  • 2.0.0
  • 1.6.1
  • 1.6.0
  • 1.5.3
  • 1.5.2
  • 1.5.1
  • 1.4.0
  • 1.3.3
  • 1.3.2
  • 1.3.1
  • 1.3.0
  • 1.2.0
  • 1.1.1
  • 1.1.0
  • 1.0.4
  • 1.0.3
  • 1.0.2
  • 1.0.1
  • 1.0.0
  • 1.0.0-pre5
  • 1.0.0-pre4
  • 1.0.0-pre3
  • 1.0.0-pre2
  • 1.0.0-pre1
  • 0.1.0
  • 0.0.5
  • 0.0.4
  • 0.0.3
  • 0.0.2
  • 0.0.1
released Oct 30th 2017

Start using this module

  • r10k or Code Manager
  • Bolt
  • Manual installation
  • Direct download

Add this module to your Puppetfile:

mod 'dalen-puppetdbquery', '3.0.1'
Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add dalen-puppetdbquery
Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install dalen-puppetdbquery --version 3.0.1

Direct download is not typically how you would use a Puppet module to manage your infrastructure, but you may want to download the module in order to inspect the code.

Download
Tags: function, puppetdb, query

Documentation

dalen/puppetdbquery — version 3.0.1 Oct 30th 2017

Build Status

PuppetDB query tools

This module implements command line tools and Puppet functions that can be used to query puppetdb. There's also a hiera backend that can be used to return query results from puppetdb.

Usage warning

You might not need this puppet module anymore. PuppetDB bundles a simplified query language since version 4.0. So unless you really prefer the syntax in this module you can just use PQL instead. See https://puppet.com/blog/introducing-puppet-query-language-pql for more details.

Requirements

PuppetDB terminus is required for the Puppet functions, but not for the face.

To parse date queries the Ruby gem "chronic" is required.

Required PuppetDB version

This module uses the V4 API, and as such it requires at least PuppetDB 3.0.0. If you are using PuppetDB 2.x please use the 1.x version of this module instead.

Query syntax

Use fact=value to search for nodes where fact equals value. To search for structured facts use dots between each part of the fact path, for example foo.bar=baz.

Resources can be matched using the syntax type[title]{param=value}. The part in brackets is optional. You can also specify ~ before the title to do a regexp match on the title. Type names and class names are case insensitive. A resource can be preceded by @@ to match exported resources, the default is to only match "local" resources.

Strings can contain letters, numbers or the characters :-_ without needing to be quoted. If they contain any other characters they need to be quoted with single or double quotes. Use backslash () to escape quotes within a quoted string or double backslash for backslashes.

An unquoted number or the strings true/false will be interpreted as numbers and boolean values, use quotation marks around them to search for them as strings instead.

A @ sign before a string causes it to be interpreted as a date parsed with chronic. For example @"2 hours ago".

A # sign can be used to do a subquery, against the nodes endpoint for example to query the report_timestamp, catalog_timestamp or facts_timestamp fields. For example #node.report_timestamp < @"2 hours ago".

A subquery using the # sign can have a block of expressions instead of a single expression. For example #node { report_timestamp > @"4 hours ago" and report_timestamp < @"2 hours ago" }

A bare string without comparison operator will be treated as a regexp match against the certname.

Comparison operators

Op Meaning
= Equality
!= Not equal
~ Regexp match
!~ Not equal Regexp match
< Less than
=< Less than or equal
> Greater than
=> Greater than or equal

Logical operators

Op
not (unary op)
and
or

Shown in precedence order from highest to lowest. Use parenthesis to change order in an expression.

Query Examples

Nodes with package mysql-server and amd64 arcitecture

(package["mysql-server"] and architecture=amd64)

Nodes with the class Postgresql::Server and a version set to 9.3

class[postgresql::server]{version=9.3}

Nodes with 4 or 8 processors running Linux

(processorcount=4 or processorcount=8) and kernel=Linux

Nodes that haven't reported in the last 2 hours

#node.report_timestamp<@"2 hours ago"

Usage

To get a list of the supported subcommands for the puppetdbquery face, run:

 $ puppet help puppetdbquery

You can run puppet help on the returned subcommands

$ puppet help puppetdbquery nodes
$ puppet help puppetdbquery facts

CLI

Each of the faces uses the following query syntax to return all objects found on a subset of nodes:

# get all nodes that contain the apache package and are in france, or all nodes in the us
$ puppet puppetdbquery nodes '(Package[httpd] and country=fr) or country=us'

Each of the individual faces returns a different data format:

nodes - a list of nodes identified by a name

 $ puppet puppetdbquery nodes '(Package["mysql-server"] and architecture=amd64)'
   ["db_node_1", "db_node2"]

facts - a hash of facts per node

 $ puppet puppetdbquery facts '(Package["mysql-server"] and architecture=amd64)'
   db_node_1  {"facterversion":"1.6.9","hostname":"controller",...........}
   db_node_2  {"facterversion":"1.6.9","hostname":"controller",...........}

events - a list of events on the matched nodes

 $ puppet puppetdbquery events '(Package["mysql-server"] and architecture=amd64)' --since='1 hour ago' --until=now --status=success
   host.example.com: 2013-06-10T10:58:37.000Z: File[/foo/bar]/content ({md5}5711edf5f5c50bd7845465471d8d39f0 -> {md5}e485e731570b8370f19a2a40489cc24b): content changed '{md5}5711edf5f5c50bd7845465471d8d39f0' to '{md5}e485e731570b8370f19a2a40489cc24b'

Ruby

faces can be called from the ruby in exactly they same way they are called from the command line:

$ irb> require 'puppet/face'
  irb> Puppet.initialize_settings
  irb> Puppet::Face[:query, :current].nodes('(Package["mysql-server"] and architecture=amd64)')

Puppet functions

There's corresponding functions to query PuppetDB directly from Puppet manifests. All the functions accept either the simplified query language or raw PuppetDB API queries.

query_nodes

Accepts two arguments, a query used to discover nodes, and a optional fact that should be returned.

Returns an array of certnames or fact values if a fact is specified.

Examples

$hosts = query_nodes('manufacturer~"Dell.*" and processorcount=24 and Class[Apache]')

$hostips = query_nodes('manufacturer~"Dell.*" and processorcount=24 and Class[Apache]', 'ipaddress')

query_resources

Accepts two arguments or three argument, a query used to discover nodes, and a resource query , and an optional a boolean to whether or not to group the result per host.

Return either a hash (by default) that maps the name of the nodes to a list of resource entries. This is a list because there's no single reliable key for resource operations that's of any use to the end user.

Examples

Returns the parameters and such for the ntp class for all CentOS nodes:

$resources = query_resources('Class["apache"]{ port = 443 }', 'User["apache"]')

Returns the parameters for the apache class for all nodes in a flat array:

query_resources(false, 'Class["apache"]', false)

query_facts

Similar to query_nodes but takes two arguments, the first is a query used to discover nodes, the second is a list of facts to return for those nodes.

Returns a nested hash where the keys are the certnames of the nodes, each containing a hash with facts and fact values.

Example

query_facts('Class[Apache]{port=443}', ['osfamily', 'ipaddress'])

Example return value in JSON format:

{
  "foo.example.com": {
    "ipaddress": "192.168.0.2",
    "osfamily": "Redhat"
  },
  "bar.example.com": {
    "ipaddress": "192.168.0.3",
    "osfamily": "Debian"
  }
}

Querying nested facts

Facter 3 introduced many nested facts, so puppetdbquery provides an easy way to query for a value nested within a fact that's a hash. To query for a nested value, simply join the keys you want to extract together on periods, like so:

Example

$host_eth0_networks = query_nodes('manufacturer~"Dell.*" and Class[Apache]', 'networking.interfaces.eth0.network')

$host_kernels_and_ips = query_facts('manufacturer~"Dell.*" and Class[Apache]', ['kernel', 'networking.interfaces.eth1.ip'])

Hiera backend

The hiera backend can be used to return an array with results from a puppetdb query. It requires another hiera backend to be active at the same time, and that will be used to define the actual puppetdb query to be used. It does not matter which backend that is, there can even be several of them. To enable add the backend puppetdbto the backends list in hiera.yaml.

hiera 3

---
:backends:
  - yaml
  - puppetdb

hiera 5

---
version: 5

hierarchy:
  - name: Puppetdb
    lookup_key: puppetdb_lookup_key

Note: hiera 5 is not backward compatible

You can not use the hiera 3 backed at all in hiera 5. Backwards compatibility is broken. You must switch to hiera 5 config to use this in hiera 5.

Examples

So instead of writing something like this in for example your hiera-data/common.yaml:

ntp::servers:
  - 'ntp1.example.com'
  - 'ntp2.example.com'

You can now instead write:

ntp::servers::_nodequery: 'Class[Ntp::Server]'

It will then find all nodes with the class ntp::server and return an array containing their certname. If you instead want to return the value of a fact, for example the ipaddress, the nodequery can be a tuple, like:

ntp::servers::_nodequery: ['Class[Ntp::Server]', 'ipaddress']

or a hash:

ntp::servers::_nodequery:
  query: 'Class[Ntp::Server]'
  fact: 'ipaddress'

Sometimes puppetdb doesn't return items in the same order every run - hiera 5 only:

ntp::servers::_nodequery: ['Class[Ntp::Server]', 'ipaddress', true]

ntp::servers::_nodequery:
  query: 'Class[Ntp::Server]'
  fact: 'ipaddress'
  sort: true

When returning facts only nodes that actually have the fact are returned, even if more nodes would in fact match the query itself.

Related projects