Version information
This version is compatible with:
- Puppet Enterprise 2023.8.x, 2023.7.x, 2023.6.x, 2023.5.x, 2023.4.x, 2023.3.x, 2023.2.x, 2023.1.x, 2023.0.x, 2021.7.x, 2021.6.x, 2021.5.x, 2021.4.x, 2021.3.x, 2021.2.x, 2021.1.x, 2021.0.x
- Puppet >= 7.0.0 < 9.0.0
- , , , ,
Start using this module
Add this module to your Puppetfile:
mod 'simp-simpkv', '0.13.0'
Learn more about managing modules with a PuppetfileDocumentation
Table of Contents
- Overview
- This is a SIMP module
- Module Description
- Setup
- Terminology
- Usage
- Reference
- Limitations
- Plugin Development
- simpkv Development
Overview
This is a SIMP module
This module is a component of the System Integrity Management Platform, a compliance-management framework built on Puppet.
If you find any issues, please submit them via JIRA.
Module Description
Provides an abstract library that allows Puppet to access one or more key/value stores.
This module provides
-
a standard Puppet language API (functions) for using key/value stores
- The API is modeled after https://github.com/docker/libkv#interface.
- See REFERENCE.md for more details on the available functions.
-
a configuration scheme that allows users to specify per-application use of different key/value store instances
-
adapter software that loads and uses store-specific interface software provided by the simpkv module itself and other modules
-
interfaces for 2 types of stores
- local filesystem store: Useful for module acceptance tests or small sites that are not subject to high-performance and/or high-availability requirements.
- LDAP store: Useful for large sites that are subject to high-performance and/or high-availability requirements.
-
support for creating custom store interfaces
- a Ruby API for the store interface software that developers can implement to provide their own store interface
- a standard acceptance testing framework developers can use to verify their store interfaces work as expected
- a developers guide.
Setup
What simpkv Affects
simpkv manages the contents of one or more key/value stores via Puppet functions which affect store, retrieve, and modify operations.
Setup Requirements
simpkv has 3 main setup requirements:
-
simpkv requires hieradata configuration (a
simpkv::options
Hash), when it is used to manage the contents of key/value stores.- simpkv will operate without any configuration by internally configuring itself to use the auto-default local filesystem store, however, this key/value store is really only useful in Puppet module tests.
-
simpkv expects each configured key/value store to be managed elsewhere.
- You can manage LDAP servers with the
simp/ds389
module.
- You can manage LDAP servers with the
-
simpkv expects any store-specific interface requirements to be addressed elsewhere.
- Some stores may impose other store-specific requirements in order for their simpkv interface software to operate. For example, a store may require packages to be installed on a server compiling the Puppet manifests.
- Any store-specific requirements will be listed in that store's interface documentation.
Terminology
The following terminology will be used throughout the remainder of this document:
-
backend- A specific key/value store that has unique configuration, (e.g., directory of files on a local filesystem, LDAP server, Consul server, Etcd server, Zookeeper server).
-
plugin - Ruby software that interfaces with a type of backend to affect the operations requested in simpkv Puppet functions. For example, the 'ldap' plugin manages simpkv data stored in an external LDAP server.
Usage
Using simpkv
is simple:
-
Use
simpkv
functions to store or retrieve key/value pairs in your Puppet code. -
Configure the backend(s) to use in hieradata.
-
Reconfigure the backend(s) in hieradata, as your needs change.
- No changes to your Puppet code will be required.
- Just transfer your data from the old key/value store to the new one.
The backend configuration of simpkv
can be as simple as you want (one backend)
or complex (multiple backends of different types servicing different
applications). Examples of both scenarios will be shown in this section.
Single Backend Example
This example will store and retrieve host information using
- simpkv function signatures that assume the default backend
- hieradata that only configures the default backend.
To store a node's hostname and IP address:
simpkv::put("hosts/${facts['clientcert']}", $facts['ipaddress'])
To create a hosts file using the list of stored host information:
$result = simpkv::list('hosts')
$result['keys'].each |$host, $info | {
host { $host:
ip => $info['value'],
}
}
In hieradata, configure the default backend in the simpkv::options
Hash. This
example, will configure simpkv's file backend.
simpkv::options:
# Hash of backend configurations.
# - We have only the required 'default' entry which will apply to
# all simpkv calls.
backends:
default:
# The plugin type and id must be specified.
type: file
id: file
# plugin-specific configuration
root_path: "/var/simp/simpkv/file"
lock_timeout_seconds: 30
Multiple Backends Example
This example will store and retrieve host information using
- simpkv function signatures that request a backend based on an application identifier
- multi-backend hieradata that supports the request.
The function signatures and hieradata are a little more complicated, but still relatively straightforward to understand.
To store a node's hostname and IP address using the backend servicing myapp1
:
$simpkv_options = { 'app_id' => 'myapp1' }
$empty_metadata = {}
simpkv::put("hosts/${facts['clientcert']}", $facts['ipaddress'], $empty_metadata, $simpkv_options)
To create a hosts file using the list of stored host information using the
backend servicing myapp1
:
$simpkv_options = { 'app_id' => 'myapp1' }
$result = simpkv::list('hosts', $simpkv_options)
$result['keys'].each |$host, $info | {
host { $host:
ip => $info['value'],
}
}
In hieradata, configure multiple backends in the simpkv::options
Hash.
This example will configure a single instance of a LDAP backend and two
instances of file backend, assuming we are using simpkv's LDAP and file plugins,
respectively. It is artificially complex, but illustrates the flexibility you
have when configuring backends:
-
You can use any mix of backend types.
-
You can use multiple instances of a backend type.
- Each 'type' and 'id' specifies a unique configuration.
-
You can map different applications to the same backend.
# The backend configurations here will be inserted into simpkv::options
# below via the alias function.
simpkv::backend::ldap_default:
type: ldap
id: default
ldap_uri: ldapi://%2fvar%2frun%2fslapd-simp_data.socket
simpkv::backend::file_myapp:
type: file
id: myapp
simpkv::backend::file_yourapp:
type: file
id: yourapp
simpkv::options:
# Hash of backend configurations.
# * Includes application-specific backends and the required default backend.
# * simpkv will use the appropriate backend for each simpkv function call.
backends:
# Backend for specific myapp_special_snowflake* applications
"myapp_special_snowflake": "%{alias('simpkv::backend::file_myapp')}"
# Backend for remaining myapp* applications, including myapp1
"myapp": "%{alias('simpkv::backend::ldap_default')}"
# Backend for all yourapp* applications
"yourapp": "%{alias('simpkv::backend::file_yourapp')}"
# Required default backend for everything else, including simpkv
# function calls with no application identifier
"default": "%{alias('simpkv::backend::ldap_default')}"
In this example, we are setting the application identifier to myapp1
in
our simpkv function calls. simpkv will select myapp
as the backend to
use for myapp1
, based on the simple matching algorithm described in
Backend Selection.
Binary Value Example
simpkv is able to store and retrieve binary values, provided the Puppet code uses the appropriate configuration and functions/types for binary data.
Below is an example of using simpkv for a binary value.
To store the content of a generated keytab file:
# Load in the binary content from a file. Returns a Binary Puppet type.
$original_binary_content = binary_file('/path/to/keytabs/app.keytab')
# Set a key/value pair with the binary content
simpkv::put('app/keytab', $original_binary_content)
To retrieve the keytab binary content and use it in a file
resource:
# Retrieve a binary value from a key/value store and set a Binary variable
$retrieved_result = simpkv::get('app/keytab')
$retrieved_binary_content = Binary.new($retrieved_result['value'], '%r')
# Persist binary data to another file
file { '/different/path/to/keytabs/app.keytab':
content => $retrieved_binary_content
}
Global Key Example
By default, the key/folder path referenced in a simpkv function is tied to the Puppet environment of the node whose manifest is being compiled. This ensures the data stored for one Puppet environment (e.g., 'dev') does not corrupt the data for another Puppet environment (e.g., 'production'). Nevertheless, there are times in which you may want to store data that is applicable to all Puppet environments, instead. simpkv supports global data through an option in each simpkv function call.
Below is an example of using simpkv to store a node's hostname and IP address as global data:
$simpkv_options = { 'global' => true }
$empty_metadata = {}
simpkv::put("hosts/${facts['clientcert']}", $facts['ipaddress'], $empty_metadata, $simpkv_options)
To create a hosts file using the list of stored, global host information:
$simpkv_options = { 'global' => true }
$result = simpkv::list('hosts', $simpkv_options)
$result['keys'].each |$host, $info | {
host { $host:
ip => $info['value'],
}
}
Auto-Default Backend
simpkv is intended to be configured via simpkv::options
and any
application-specific configuration passed to the simpkv Puppet functions.
However, to facilitate Puppet manifest testing and the rollout of simpkv
capabilities (specifically, the use of simpkv internally in simplib::passgen
),
when simpkv::options
is not set in hieradata, simpkv will automatically use
the 'file' plugin to store key/value data on the local filesystem. This is
equivalent to the following simpkv::options
hieradata:
simpkv::options:
environment: "%{server_facts.environment}"
softfail: false
backend: default
backends:
default:
type: file
id: auto_default
The 'file' plugin id described here.
Reference
simpkv Function Reference
See REFERENCE.md for the module's function reference.
simpkv Configuration Reference
The simpkv configuration used for each simpkv function call is comprised of a merge of
- function-provided options Hash
- hieradata configuration specified by the
simpkv::options
Hash - global configuration defaults.
The merge is executed in a fashion to ensure the function-provided options take
precedence over the simpkv::options
hieradata values and global defaults.
The merged simpkv configuration contains global and backend-specific configurations, along with an optional, function-provided application identifier. The primary keys in this Hash are as follows:
-
app_id
: Optional String from simpkv function calls, only. Specifies an application name that can be used to identify which backend configuration to use via fuzzy name matching, in the absence of thebackend
option.- More flexible option than
backend
. - Useful for grouping together simpkv function calls found in different catalog resources.
- See Backend Selection.
- More flexible option than
-
backend
: Optional String. Specifies a definitive backend configuration to use.-
Takes precedence over
app_id
. -
When present, must match a key in
backends
and will be used unequivocally.- If that backend does not exist in
backends
, the simpkv function will fail.
- If that backend does not exist in
-
When absent, the backend configuration will be selected from the set of entries in
backends
, using theapp_id
option if specified. -
See Backend Selection.
-
-
backends
: Required Hash. Specifies backend configurations. Each key is the name of a backend configuration and its value contains the corresponding configuration Hash.- Each key is a String.
- Must include a 'default' key.
- More than one key can use the same backend configuration.
- See Backend Configuration Entries for more details about a backend configuration Hash.
-
global
: Optional Boolean. Set totrue
when the key being accessed is global. Otherwise, the key will be tied to the Puppet environment of the node whose manifest is being compiled.- Defaults to
false
.
- Defaults to
-
softfail
: Optional Boolean. Whether to ignore simpkv operation failures.- When
true
, each simpkv function will return a result object even when the operation failed at the backend. - When
false
, each simpkv function will fail when the backend operation failed. - Defaults to
false
when absent.
- When
Backend Configuration Entries
Each backend configuration entry in backends
is a Hash. The Hash must
contain type
and id
keys, where the (type
,id
) pair defines a unique
configuration (i.e., unique plugin instance).
-
type
is the type of plugin, e.g., 'ldap'.- 'file' and 'ldap' are provided by the simpkv module.
- Other modules may provide their own plugins.
-
id
is the instance identifier for atype
. -
Other keys for configuration specific to the backend may also be present.
Backend Selection
The backend to use for a simpkv Puppet function call will be determined from the merged simpkv options Hash as follows:
-
If a specific backend is requested via the
backend
key in the merged simpkv options Hash, that backend will be selected.- If that backend does not exist in
backends
, the simpkv function will fail.
- If that backend does not exist in
-
Otherwise, if an
app_id
option is specified in the merged simpkv options Hash and it matches a key in thebackends
Hash, exactly, that backend will be selected. -
Otherwise, if an
app_id
option is specified in the merged simpkv options Hash and it starts with the key in thebackends
Hash, that backend will be selected.- When multiple backends satisfy the 'start with' match, the backend with the most matching characters is selected.
-
Otherwise, if the
app_id
option does not match any key in in thebackends
Hash or is not present, thedefault
backend will be selected.
To clarify this backend selection algorithm, consider a site in which simpkv is
configured for multiple backends in the following simpkv::options
Hash.
simpkv::options:
# Hash of backend configurations.
# * Includes application-specific backends and the required default backend.
# * simpkv will use the appropriate backend for each simpkv function call.
backends:
# Backend for specific myapp_special_snowflake* applications
"myapp_special_snowflake":
type: file
id: myapp_special_snowflake
# Backend for remaining myapp* applications
"myapp":
type: file
id: myapp
# Backend for all yourapp* applications
"yourapp":
type: file
id: yourapp
# required default backend for everything else, including simpkv
# function calls with no application identifier
"default":
type: file
id: default
The following table shows the backend that simpkv will select make given the options specified by the simpkv function call.
simpkv Function Call | Backend Selected | Comments |
---|---|---|
simpkv::get('key') |
default |
No special handling requested |
simpkv::get('key', {'backend' => 'yourapp'}) |
yourapp |
Specific backend requested |
simpkv::get('key', {'backend' => 'oops'}) |
N/A | Function will fail since 'oops' backend does not exist |
simpkv::get('key', {'app_id' => 'myapp'}) |
myapp |
Exact match |
simpkv::get('key', {'app_id' => 'myapp1'}) |
myapp |
Starts with match |
simpkv::get('key', {'app_id' => 'myapp_special'}) |
myapp |
Starts with match |
simpkv::get('key', {'app_id' => 'myapp_special_snowflake_for_bob'}) |
myapp_special_snowflake |
Starts with match |
simpkv::get('key', {'app_id' => 'otherapp'}) |
default |
No match so fallback to default |
Backend Basics
This section describes details about the folder layout and format of data stored by simpkv that you may find useful when you are inspecting data in a backend.
Backend Folder Layout
The storage in a simpkv backend can be notionally represented as a folder tree with key files at terminal nodes. simpkv automatically sets up the folder layout at the top level and the user specifies key files below that. Specifically,
-
simpkv stores global keys in a
globals
sub-folder of the root folder.- Global keys are not tied to any specific Puppet environment.
- You must specify
'global' => true
in the options passed to simpkv functions in order to access global keys.
-
simpkv stores all other keys in sub-folders named for the Puppet environment in which each key was created.
- The parent directory for all environment folders is
<root folder>/environments
.
- The parent directory for all environment folders is
-
Further sub-folder trees are allowed for global or environment-specific keys.
- A relative paths in a key name indicates a sub-folder tree (e.g.
'app1/keya'
).
- A relative paths in a key name indicates a sub-folder tree (e.g.
For example,
The actual representation of the root folder and 'key file' is backend specific.
Backend Data Format
Internally, simpkv automatically serializes each key's value and optional metadata into a string for backend storage, and then deserializes it upon retrieval. The string is a JSON representation of a Hash with at least 2 attributes:
-
value
: Always present. Key's value -
metadata
: Always present. Key's metadata -
encoding
andoriginal_encoding
: Only present when the key's value is a Puppet Binary type. Indicates simpkv's internal encoding of the binary data into a representation suitable for JSON.- simpkv uses strict Base64 encoding.
This serialization simplifies plugin development, but does limit the types of data that can be stored in the value and metadata. simpkv works with the following types:
- Simple type: String, boolean, or numeric values
- Standard complex type: Hash, Array, or nested Hash/Array structure whose terminal nodes are simple types
- Binary type: Puppet Binary type when it is the key's value, only; not in any complex values or metadata
The table below shows a few examples of the serialization for clarification.
Value Type | Serialized String |
---|---|
Simple without metadata | {"value":10,"metadata":{}} |
Simple with metadata | {"value":true,"metadata":{"verified":true,"user":"vsmith"}} |
Array with metadata | {"value":[1,2,3],"metadata":{"originator":"njones","location":{"room":"29B","rack":10}}} |
Hash without metadata | {"value":{"attr1":"hello","attr2":{"part1":9.898,"part2":[1,2,3]}},"metadata":{} |
Binary transformed by simpkv without metadata | {"value":"<Base64 (strict) encoded string>","encoding":"base64","original_encoding":"ASCII-8BIT","metadata":{}"} |
File Plugin
simpkv provides a plugin that maintains a file-based key/value store on the local filesystem on the server on which the Puppet manifests are being compiled. It effectively uses system commands to affect local filesystems changes.
- This plugin is appropriate for Puppet module tests.
- This plugin can be used for small-sites in which there is only a single server compiling Puppet manifests.
File Plugin Requirements
-
The file plugin must be configured for a local filesystem on the server compiling the Puppet manifests.
- The file locking mechanism the plugin uses to ensure the integrity of the key files is only guaranteed to work on a local filesystem, and is not suitable for shared filesystems such as NFS.
-
The file plugin must have write privileges for the configured
root_path
for the user compiling the manifests. -
The file plugin must not be used for sites using distributed Puppet servers.
- The file plugin has no mechanisms to distribute the key/store to other compile servers.
File Plugin Configuration
The plugin has a backend type file
, and supports the following plugin-specific
configuration parameters.
-
root_path
: Optional. Root directory path for the key files- Defaults to
/var/simp/simpkv/file/<id>
when that directory can be created or<Puppet[:vardir]>/simp/simpkv/<name>
otherwise.
- Defaults to
-
lock_timeout_seconds
: Optional. Maximum number of seconds to wait for an exclusive file lock on a file modifying operation before failing the operation.- Defaults to 5 seconds.
Here is an example configuration for the file plugin:
simpkv::options:
backends:
default:
type: file
id: default
LDAP Plugin
simpkv provides a plugin to interface with a LDAP key/value store configured with a simpkv LDAP schema. If your site is large and requires a high-availability, distributed key/value store, LDAP is the appropriate backend to use! The benefits of using LDAP are as follows:
- LDAP supports the storage of any data, not just accounts data.
- LDAP is well defined protocol implemented by a wide variety of client and server implementations.
- LDAP is secure.
- LDAP server implementations are highly performant.
- LDAP server implementations support replication.
LDAP Plugin Requirements
The LDAP plugin has 4 main requirements:
-
The package providing
ldapadd
,ldapdelete
,ldapmodify
andldapsearch
(i.e.openldap-clients
) must be installed on the system on which the Puppet manifests will be compiled. -
The LDAP server must be loaded with the simpkv LDAP schema.
-
The root Directory Name (DN) for the simpkv tree in LDAP must already exist.
- This corresponds to the LDAP plugin's
base_dn
configuration parameter.
- This corresponds to the LDAP plugin's
-
The supplied LDAP configuration must permit the user compiling the Puppet manifests to modify the LDAP tree below that 'simpkv' root DN.
In addition, if LDAPI is not the protocol being being used to communicate with the LDAP server, the configured password file used for simple authentication with the LDAP server MUST already exist before the catalog is compiled.
- Catalog compilation will fail if you attempt to create the file using a
file
Puppet resource in the same catalog as simpkv function calls. This is because the simpkv functions will be evaluated before thefile
resource in the compilation.
LDAP Plugin Configuration
The simpkv LDAP plugin is of type ldap
and supports the following
plugin-specific configuration parameters.
-
ldap_uri
: Required. The LDAP server URI.- This can be a LDAPI socket path or an ldap/ldaps URI specifying host and, optionally, port.
- When using an 'ldap://' URI with StartTLS,
enable_tls
must be true andtls_cert
,tls_key
, andtls_cacert
must be configured. - When using an 'ldaps://' URI,
tls_cert
,tls_key
, andtls_cacert
must be configured.
-
base_dn
: Optional. The root DN for the 'simpkv' tree in LDAP.- Defaults to 'ou=simpkv,o=puppet,dc=simp'
- Must already exist
-
admin_dn
: Optional. The bind DN for simpkv administration.- Defaults to 'cn=Directory_Manager'
- This identity must have permission to modify the LDAP tree below
base_dn
.
-
admin_pw_file
: Required for all but LDAPI. A file containing the simpkv adminstration password.- Will be used for authentication when set, even with LDAPI.
- When unset for LDAPI, the
admin_dn
is assumed to be properly configured for external EXTERNAL SASL authentication for the user compiling the manifest (e.g., 'puppet' for 'puppet agent', 'root' for 'puppet apply' and the Bolt user for Bolt plans).
-
enable_tls
: Optional. Whether to enable TLS.- Defaults to true when
ldap_uri
is an 'ldaps://' URI, otherwise defaults to false. - Must be set to true to enable StartTLS when using an 'ldap://' URI.
- When true
tls_cert
,tls_key
andtls_cacert
must be set.
- Defaults to true when
-
tls_cert
: Required for StartTLS or TLS. The certificate file. -
tls_key
: Required for StartTLS or TLS. The key file. -
tls_cacert
: Required for StartTLS or TLS. The cacert file. -
retries
: Optional. Number of times to retry an LDAP operation if the server reports it is busy.- Defaults to 1.
Here are some example configurations:
- To use LDAPI (best option if the Puppet manifest compilation is being done on the LDAP server), configure the LDAP plugin as follows:
simpkv::options:
backends:
default:
type: ldap
id: default
# Set this to your LDAP server's LDAPI URI
ldap_uri: ldapi://%2fvar%2frun%2fslapd-simp_data.socket
- To use LDAPS, configure the LDAP plugin as follows:
simpkv::options:
backends:
default:
type: ldap
id: default
# Set this to your LDAP server's LDAPS URI
ldap_uri: ldaps://ldap.example.com
# Set this to the location of your administration DN password file
admin_pw_file: /etc/simp/simpkv_pw.txt
# Set these to the appropriate certs for your server
tls_cert: /etc/pki/simp_apps/openldap/public/puppetserver.example.com.pub
tls_key: /etc/pki/simp_apps/openldap/private/puppetserver.example.com.pem
tls_cacert: /etc/pki/simp_apps/openldap/cacerts/cacerts.pem
- To use LDAP with StartTLS, configure the LDAP plugin as follows:
simpkv::options:
backends:
default:
type: ldap
id: default
# Set this to your LDAP server's LDAP URI
ldap_uri: ldap://ldap.example.com
# Set this to the location of your administration DN password file
admin_pw_file: /etc/simp/simpkv_pw.txt
# Use StartTLS with the LDAP URI
enable_tls: true
# Set these to the appropriate certs for your server
tls_cert: /etc/pki/simp_apps/openldap/public/puppetserver.example.com.pub
tls_key: /etc/pki/simp_apps/openldap/private/puppetserver.example.com.pem
tls_cacert: /etc/pki/simp_apps/openldap/cacerts/cacerts.pem
The LDAP plugin creates the key/value store for each instance under its own
tree on the LDAP server. The tree's DN is ou=,ou=instances,<base_dn>,
where the instance id is the 'ldap' plugin's id
configuration parameter. So,
you can actually use the same LDAP server for different plugin instances.
Here is an example configuration of multiple instances with the same LDAPI configuration:
simpkv::options:
backends:
# stored under the 'myapp' instance tree of the simp_data LDAP server
myapp:
type: ldap
id: myapp
ldap_uri: ldapi://%2fvar%2frun%2fslapd-simp_data.socket
# stored under the 'default' instance tree of the simp_data LDAP server
default:
type: ldap
id: default
ldap_uri: ldapi://%2fvar%2frun%2fslapd-simp_data.socket
Debugging LDAP Integration Issues
When you are trying to debug LDAP integration issues, use the --debug
option during manifest compilation to see more details about what the
LDAP plugin is doing. For example,
puppet apply -t --debug my_manifests.pp
Using this option will log the exact LDAP commands used during simpkv function calls.
Other Resources
The ldap_plugin
acceptance test suite is complete example on how to use the
LDAP plugin. It demonstrates the following:
-
How to configure simpkv to use the LDAP plugin for LDAPI unencrypted LDAP, LDAP + StartTLS, and LDAPS protocols.
-
How to stand up simple 389-DS LDAP instances for those protocol options.
Each 389-DS instance is
-
Created by the
simp/ds389
module. -
Bootstrapped with a LDIF that corresponds to the simpkv leg of the SIMP Data Root Directory Information Tree (DIT) shown below.
-
Explicitly configured with non-standard ports, so as to not be confused with LDAP servers used for accounts management.
-
Loaded with the simpkv LDAP schema.
-
Limitations
-
SIMP Puppet modules are generally intended to be used on a Red Hat Enterprise Linux-compatible distribution such as EL7 and EL8.
-
simpkv
only supports the use of binary data for the value when that data is a PuppetBinary
. It does not support binary data which is a sub-element of a more complex value type (e.g.Array[Binary]
orHash
that has a key or value that is aBinary
). -
simpkv
does not support custom data types that cannot be serialized to JSON by the Ruby libraries provided by Puppet. -
simpkv's file plugin is only guaranteed to work on local filesystems. It may not work on shared filesystems, such as NFS.
-
SIMP does not yet manage the LDAP server configured for simpkv data.
- See the
ldap_plugin
acceptance test suite for an example of how to set up a simple 389-DS LDAP instance for simpkv data.
- See the
Plugin Development
Please see simpkv Plugin Development Guide for details on how to write your own custom plugin.
simpkv Development
Please read our [Contribution Guide] (https://simp.readthedocs.io/en/stable/contributors_guide/index.html).
Unit tests
Unit tests, written in rspec-puppet
can be run by calling:
bundle install
bundle exec rake spec
Acceptance tests
This module includes Beaker acceptance tests using the SIMP Beaker Helpers. By default the tests use Vagrant with VirtualBox as a back-end; Vagrant and VirtualBox must both be installed to run these tests without modification.
To execute all of the tests run the following:
bundle install
bundle exec rake beaker:suites
To execute an individual suite for a specific nodeset, add the suite and the
nodeset to the beaker:suites
task. For example,
bundle exec rake beaker:suites[ldap_plugin,default]
Some environment variables that may be useful:
BEAKER_debug=true
BEAKER_destroy=no
BEAKER_fips=yes
BEAKER_provision=no
-
BEAKER_debug
: Show the commands being run on the virtual machines and their output. -
BEAKER_destroy=no
: Prevent the virtual machine destruction after the tests finish so you can inspect the state. -
BEAKER_fips=yes
: Enable FIPS-mode on the virtual machines.- This can take a long time, because it must enable FIPS in the kernel command-line, rebuild the initramfs, then reboot.
-
BEAKER_provision=no
: Prevent the virtual machine from being recreated.- This can save a lot of time while you're writing the tests.
Please refer to the SIMP Beaker Helpers documentation for more information.
Reference
Table of Contents
Functions
simpkv::delete
: Deletes akey
from the configured backend.simpkv::deletetree
: Deletes a whole folder from the configured backend.simpkv::exists
: Returns whether key or key folder exists in the configured backend.simpkv::get
: Retrieves the value and any metadata stored atkey
from the configured backend.simpkv::list
: Returns a listing of all keys and sub-folders in a folder. The list operation does not recurse through any sub-folders. Only information abosimpkv::put
: Sets the data atkey
to the specifiedvalue
in the configured backend. Optionally sets metadata along with thevalue
.simpkv::support::config::merge
: Create merged backend configuration and then validate it. The merge entails the following operations: * Theoptions
argument is merged witsimpkv::support::config::validate
: Validate backend configurationsimpkv::support::key::validate
: Validates key conforms to the simpkv key specification simpkv key specification Key must contain only the following characters:simpkv::support::load
: Load simpkv adapter and plugins and add simpkv 'extension' to the catalog instance, if it is not present
Functions
simpkv::delete
Type: Ruby 4.x API
Deletes a key
from the configured backend.
Examples
Delete a key from the default backend
simpkv::delete("hosts/${facts['fqdn']}")
Delete a key from the backend servicing an application id
simpkv::delete("hosts/${facts['fqdn']}", { 'app_id' => 'myapp' })
Delete a global key from the default backend
simpkv::delete("hosts/${facts['fqdn']}", { 'global' => true })
simpkv::delete(String[1] $key, Optional[Hash] $options)
Deletes a key
from the configured backend.
Returns: Boolean
true
when backend operation succeeds; false
when the
backend operation fails and 'softfail' is true
in the merged backend
options
Raises:
ArgumentError
If the key or merged backend config is invalidLoadError
If the simpkv adapter cannot be loadedRuntimeError
If the backend operation fails, unless 'softfail' istrue
in the merged backend options.
Examples
Delete a key from the default backend
simpkv::delete("hosts/${facts['fqdn']}")
Delete a key from the backend servicing an application id
simpkv::delete("hosts/${facts['fqdn']}", { 'app_id' => 'myapp' })
Delete a global key from the default backend
simpkv::delete("hosts/${facts['fqdn']}", { 'global' => true })
key
Data type: String[1]
The key to remove. Must conform to the following:
-
Key must contain only the following characters:
- a-z
- 0-9
- The following special characters:
._:-/
-
Key may not contain '/./' or '/../' sequences.
options
Data type: Optional[Hash]
simpkv configuration that will be merged with
simpkv::options
. All keys are optional.
Options:
-
'app_id'
String
: Specifies an application name that can be used to identify which backend configuration to use via fuzzy name matching, in the absence of thebackend
option.- More flexible option than
backend
. - Useful for grouping together simpkv function calls found in different catalog resources.
- When specified and the
backend
option is absent, the backend will be selected preferring a backend in the mergedbackends
option whose name exactly matches theapp_id
, followed by the longest backend name that matches the beginning of theapp_id
, followed by thedefault
backend. - When absent and the
backend
option is also absent, this function will use thedefault
backend.
- More flexible option than
-
'backend'
String
: Definitive name of the backend to use.- Takes precedence over
app_id
. - When present, must match a key in the
backends
option of the merged options Hash or the function will fail. - When absent in the merged options, this function will select
the backend as described in the
app_id
option.
- Takes precedence over
-
'backends'
Hash
: Hash of backend configurations-
Each backend configuration in the merged options Hash must be a Hash that has the following keys:
type
: Backend type.id
: Unique name for the instance of the backend. (Same backend type can be configured differently).
-
Other keys for configuration specific to the backend may also be present.
-
-
'global'
Boolean
: Set totrue
when the key being accessed is global. Otherwise, the key will be tied to the Puppet environment of the node whose manifest is being compiled.- Defaults to
false
- Defaults to
-
'softfail'
Boolean
: Whether to ignore simpkv operation failures.- When
true
, this function will return a result even when the operation failed at the backend. - When
false
, this function will fail when the backend operation failed. - Defaults to
false
.
- When
simpkv::deletetree
Type: Ruby 4.x API
Deletes a whole folder from the configured backend.
Examples
Delete a key folder from the default backend
simpkv::deletetree("hosts")
Delete a key folder from the backend servicing an application id
simpkv::deletetree("hosts", { 'app_id' => 'myapp' })
Delete a global key folder from the default backend
simpkv::deletetree("hosts", { 'global' => true })
simpkv::deletetree(String[1] $keydir, Optional[Hash] $options)
Deletes a whole folder from the configured backend.
Returns: Boolean
true
when backend operation succeeds; false
when the
backend operation fails and 'softfail' is true
in the merged backend
options
Raises:
ArgumentError
If the key folder or merged backend config is invalidLoadError
If the simpkv adapter cannot be loadedRuntimeError
If the backend operation fails, unless 'softfail' istrue
in the merged backend options.
Examples
Delete a key folder from the default backend
simpkv::deletetree("hosts")
Delete a key folder from the backend servicing an application id
simpkv::deletetree("hosts", { 'app_id' => 'myapp' })
Delete a global key folder from the default backend
simpkv::deletetree("hosts", { 'global' => true })
keydir
Data type: String[1]
The key folder to remove. Must conform to the following:
-
Key folder must contain only the following characters:
- a-z
- 0-9
- The following special characters:
._:-/
-
Key folder may not contain '/./' or '/../' sequences.
options
Data type: Optional[Hash]
simpkv configuration that will be merged with
simpkv::options
. All keys are optional.
Options:
-
'app_id'
String
: Specifies an application name that can be used to identify which backend configuration to use via fuzzy name matching, in the absence of thebackend
option.- More flexible option than
backend
. - Useful for grouping together simpkv function calls found in different catalog resources.
- When specified and the
backend
option is absent, the backend will be selected preferring a backend in the mergedbackends
option whose name exactly matches theapp_id
, followed by the longest backend name that matches the beginning of theapp_id
, followed by thedefault
backend. - When absent and the
backend
option is also absent, this function will use thedefault
backend.
- More flexible option than
-
'backend'
String
: Definitive name of the backend to use.- Takes precedence over
app_id
. - When present, must match a key in the
backends
option of the merged options Hash or the function will fail. - When absent in the merged options, this function will select
the backend as described in the
app_id
option.
- Takes precedence over
-
'backends'
Hash
: Hash of backend configurations-
Each backend configuration in the merged options Hash must be a Hash that has the following keys:
type
: Backend type.id
: Unique name for the instance of the backend. (Same backend type can be configured differently).
-
Other keys for configuration specific to the backend may also be present.
-
-
'global'
Boolean
: Set totrue
when the key being accessed is global. Otherwise, the key will be tied to the Puppet environment of the node whose manifest is being compiled.- Defaults to
false
- Defaults to
-
'softfail'
Boolean
: Whether to ignore simpkv operation failures.- When
true
, this function will return a result even when the operation failed at the backend. - When
false
, this function will fail when the backend operation failed. - Defaults to
false
.
- When
simpkv::exists
Type: Ruby 4.x API
Returns whether key or key folder exists in the configured backend.
Examples
Check for the existence of a key in the default backend
if simpkv::exists("hosts/${facts['fqdn']}") {
notify { "hosts/${facts['fqdn']} exists": }
}
Check for the existence of a key in the backend servicing an application id
if simpkv::exists("hosts/${facts['fqdn']}", { 'app_id' => 'myapp' }) {
notify { "hosts/${facts['fqdn']} exists": }
}
Check for the existence of a global key folder in the default backend
if simpkv::exists("hosts", { 'global' => true}) {
notify { 'hosts folder exists': }
}
simpkv::exists(String[1] $key, Optional[Hash] $options)
Returns whether key or key folder exists in the configured backend.
Returns: Enum[Boolean,Undef]
If the backend operation succeeds, returns
true
or false
; if the backend operation fails and 'softfail' is true
in the merged backend options, returns nil
Raises:
ArgumentError
If the key or merged backend config is invalidLoadError
If the simpkv adapter cannot be loadedRuntimeError
If the backend operation fails, unless 'softfail' istrue
in the merged backend options.
Examples
Check for the existence of a key in the default backend
if simpkv::exists("hosts/${facts['fqdn']}") {
notify { "hosts/${facts['fqdn']} exists": }
}
Check for the existence of a key in the backend servicing an application id
if simpkv::exists("hosts/${facts['fqdn']}", { 'app_id' => 'myapp' }) {
notify { "hosts/${facts['fqdn']} exists": }
}
Check for the existence of a global key folder in the default backend
if simpkv::exists("hosts", { 'global' => true}) {
notify { 'hosts folder exists': }
}
key
Data type: String[1]
The key or key folder to check. Must conform to the following:
-
Key must contain only the following characters:
- a-z
- 0-9
- The following special characters:
._:-/
-
Key may not contain '/./' or '/../' sequences.
options
Data type: Optional[Hash]
simpkv configuration that will be merged with
simpkv::options
. All keys are optional.
Options:
-
'app_id'
String
: Specifies an application name that can be used to identify which backend configuration to use via fuzzy name matching, in the absence of thebackend
option.- More flexible option than
backend
. - Useful for grouping together simpkv function calls found in different catalog resources.
- When specified and the
backend
option is absent, the backend will be selected preferring a backend in the mergedbackends
option whose name exactly matches theapp_id
, followed by the longest backend name that matches the beginning of theapp_id
, followed by thedefault
backend. - When absent and the
backend
option is also absent, this function will use thedefault
backend.
- More flexible option than
-
'backend'
String
: Definitive name of the backend to use.- Takes precedence over
app_id
. - When present, must match a key in the
backends
option of the merged options Hash or the function will fail. - When absent in the merged options, this function will select
the backend as described in the
app_id
option.
- Takes precedence over
-
'backends'
Hash
: Hash of backend configurations-
Each backend configuration in the merged options Hash must be a Hash that has the following keys:
type
: Backend type.id
: Unique name for the instance of the backend. (Same backend type can be configured differently).
-
Other keys for configuration specific to the backend may also be present.
-
-
'global'
Boolean
: Set totrue
when the key being accessed is global. Otherwise, the key will be tied to the Puppet environment of the node whose manifest is being compiled.- Defaults to
false
- Defaults to
-
'softfail'
Boolean
: Whether to ignore simpkv operation failures.- When
true
, this function will return a result even when the operation failed at the backend. - When
false
, this function will fail when the backend operation failed. - Defaults to
false
.
- When
simpkv::get
Type: Ruby 4.x API
Retrieves the value and any metadata stored at key
from the
configured backend.
Examples
Retrieve the value and any metadata for a key from the default backend
$result = simpkv::get("database/${facts['fqdn']}")
class { 'wordpress':
db_host => $result['value'],
db_info => $result['metadata']
}
Retrieve the value and any metadata for a key from the backend servicing an application id
$result = simpkv::get("database/${facts['fqdn']}", { 'app_id' => 'myapp' })
class { 'wordpress':
db_host => $result['value'],
db_info => $result['metadata']
}
Retrieve the value and any metadata for a global key from the default backend
$result = simpkv::get("database/${facts['fqdn']}", { 'global' => true })
class { 'wordpress':
db_host => $result['value'],
db_info => $result['metadata']
}
simpkv::get(String[1] $key, Optional[Hash] $options)
Retrieves the value and any metadata stored at key
from the
configured backend.
Returns: Enum[Hash,Undef]
Hash containing the value and any metadata upon
success; Undef when the backend operation fails and 'softfail' is true
in the merged backend options
- Hash will have a 'value' key containing the retrieved value
- Hash may have a 'metadata' key containing a Hash with any metadata for the key
Raises:
ArgumentError
If the key or merged backend config is invalidLoadError
If the simpkv adapter cannot be loadedRuntimeError
If the backend operation fails, unless 'softfail' istrue
in the merged backend options.
Examples
Retrieve the value and any metadata for a key from the default backend
$result = simpkv::get("database/${facts['fqdn']}")
class { 'wordpress':
db_host => $result['value'],
db_info => $result['metadata']
}
Retrieve the value and any metadata for a key from the backend servicing an application id
$result = simpkv::get("database/${facts['fqdn']}", { 'app_id' => 'myapp' })
class { 'wordpress':
db_host => $result['value'],
db_info => $result['metadata']
}
Retrieve the value and any metadata for a global key from the default backend
$result = simpkv::get("database/${facts['fqdn']}", { 'global' => true })
class { 'wordpress':
db_host => $result['value'],
db_info => $result['metadata']
}
key
Data type: String[1]
The key to retrieve. Must conform to the following:
-
Key must contain only the following characters:
- a-z
- 0-9
- The following special characters:
._:-/
-
Key may not contain '/./' or '/../' sequences.
options
Data type: Optional[Hash]
simpkv configuration that will be merged with
simpkv::options
. All keys are optional.
Options:
-
'app_id'
String
: Specifies an application name that can be used to identify which backend configuration to use via fuzzy name matching, in the absence of thebackend
option.- More flexible option than
backend
. - Useful for grouping together simpkv function calls found in different catalog resources.
- When specified and the
backend
option is absent, the backend will be selected preferring a backend in the mergedbackends
option whose name exactly matches theapp_id
, followed by the longest backend name that matches the beginning of theapp_id
, followed by thedefault
backend. - When absent and the
backend
option is also absent, this function will use thedefault
backend.
- More flexible option than
-
'backend'
String
: Definitive name of the backend to use.- Takes precedence over
app_id
. - When present, must match a key in the
backends
option of the merged options Hash or the function will fail. - When absent in the merged options, this function will select
the backend as described in the
app_id
option.
- Takes precedence over
-
'backends'
Hash
: Hash of backend configurations-
Each backend configuration in the merged options Hash must be a Hash that has the following keys:
type
: Backend type.id
: Unique name for the instance of the backend. (Same backend type can be configured differently).
-
Other keys for configuration specific to the backend may also be present.
-
-
'global'
Boolean
: Set totrue
when the key being accessed is global. Otherwise, the key will be tied to the Puppet environment of the node whose manifest is being compiled.- Defaults to
false
- Defaults to
-
'softfail'
Boolean
: Whether to ignore simpkv operation failures.- When
true
, this function will return a result even when the operation failed at the backend. - When
false
, this function will fail when the backend operation failed. - Defaults to
false
.
- When
simpkv::list
Type: Ruby 4.x API
Returns a listing of all keys and sub-folders in a folder.
The list operation does not recurse through any sub-folders. Only information about the specified key folder is returned.
Examples
Retrieve the list of key info for a key folder from the default backend
$result = simpkv::list('hosts')
$result['keys'].each |$host, $info | {
host { $host:
ip => $info['value'],
}
}
Retrieve the list of sub-folders in a key folder from the backend servicing an application id
$result = simpkv::list('applications', { 'app_id' => 'myapp' })
notice("Supported applications: ${join($result['folders'], ' ')}")
Retrieve the top level list of global keys/folders in the default backend
$result = simpkv::list('/', { 'global' = true })
notice("Global folders: ${join($result['folders'], ' ')}")
$result['keys'].each |$key, $info | {
notice("Global key ${key}: value=${info['value']} metadata=${info['metadata']}")
}
simpkv::list(String[1] $keydir, Optional[Hash] $options)
Returns a listing of all keys and sub-folders in a folder.
The list operation does not recurse through any sub-folders. Only information about the specified key folder is returned.
Returns: Enum[Hash,Undef]
Hash containing the key/info pairs and list of
sub-folders upon success; Undef when the backend operation fails and
'softfail' is true
in the merged backend options
- 'keys' attribute is a Hash of the key information in the folder
- Each Hash key is a key found in the folder
- Each Hash value is a Hash with a 'value' key and an optional 'metadata' key.
- 'folders' attribute is an Array of sub-folder names
Raises:
ArgumentError
If the key folder or merged backend config is invalidLoadError
If the simpkv adapter cannot be loadedRuntimeError
If the backend operation fails, unless 'softfail' istrue
in the merged backend options.
Examples
Retrieve the list of key info for a key folder from the default backend
$result = simpkv::list('hosts')
$result['keys'].each |$host, $info | {
host { $host:
ip => $info['value'],
}
}
Retrieve the list of sub-folders in a key folder from the backend servicing an application id
$result = simpkv::list('applications', { 'app_id' => 'myapp' })
notice("Supported applications: ${join($result['folders'], ' ')}")
Retrieve the top level list of global keys/folders in the default backend
$result = simpkv::list('/', { 'global' = true })
notice("Global folders: ${join($result['folders'], ' ')}")
$result['keys'].each |$key, $info | {
notice("Global key ${key}: value=${info['value']} metadata=${info['metadata']}")
}
keydir
Data type: String[1]
The key folder to list. Must conform to the following:
-
Key folder must contain only the following characters:
- a-z
- 0-9
- The following special characters:
._:-/
-
Key folder may not contain '/./' or '/../' sequences.
options
Data type: Optional[Hash]
simpkv configuration that will be merged with
simpkv::options
. All keys are optional.
Options:
-
'app_id'
String
: Specifies an application name that can be used to identify which backend configuration to use via fuzzy name matching, in the absence of thebackend
option.- More flexible option than
backend
. - Useful for grouping together simpkv function calls found in different catalog resources.
- When specified and the
backend
option is absent, the backend will be selected preferring a backend in the mergedbackends
option whose name exactly matches theapp_id
, followed by the longest backend name that matches the beginning of theapp_id
, followed by thedefault
backend. - When absent and the
backend
option is also absent, this function will use thedefault
backend.
- More flexible option than
-
'backend'
String
: Definitive name of the backend to use.- Takes precedence over
app_id
. - When present, must match a key in the
backends
option of the merged options Hash or the function will fail. - When absent in the merged options, this function will select
the backend as described in the
app_id
option.
- Takes precedence over
-
'backends'
Hash
: Hash of backend configurations-
Each backend configuration in the merged options Hash must be a Hash that has the following keys:
type
: Backend type.id
: Unique name for the instance of the backend. (Same backend type can be configured differently).
-
Other keys for configuration specific to the backend may also be present.
-
-
'global'
Boolean
: Set totrue
when the key being accessed is global. Otherwise, the key will be tied to the Puppet environment of the node whose manifest is being compiled.- Defaults to
false
- Defaults to
-
'softfail'
Boolean
: Whether to ignore simpkv operation failures.- When
true
, this function will return a result even when the operation failed at the backend. - When
false
, this function will fail when the backend operation failed. - Defaults to
false
.
- When
simpkv::put
Type: Ruby 4.x API
Sets the data at key
to the specified value
in the configured backend.
Optionally sets metadata along with the value
.
Examples
Set a key in the default backend
simpkv::put("hosts/${facts['clientcert']}", $facts['ipaddress'])
Set a key with metadata in the backend servicing an application id
$meta = { 'rack_id' => 183 }
$opts = { 'app_id' => 'myapp' }
simpkv::put("hosts/${facts['clientcert']}", $facts['ipaddress'], $meta, $opts)
Set a gobal key in the default backend
simpkv::put("hosts/${facts['clientcert']}", $facts['ipaddress'], { 'global' => true })
simpkv::put(String[1] $key, NotUndef $value, Optional[Hash] $metadata, Optional[Hash] $options)
Sets the data at key
to the specified value
in the configured backend.
Optionally sets metadata along with the value
.
Returns: Boolean
true
when backend operation succeeds; false
when the
backend operation fails and 'softfail' is true
in the merged backend
options
Raises:
ArgumentError
If the key or merged backend config is invalidLoadError
If the simpkv adapter cannot be loadedRuntimeError
If the backend operation fails, unless 'softfail' istrue
in the merged backend options.
Examples
Set a key in the default backend
simpkv::put("hosts/${facts['clientcert']}", $facts['ipaddress'])
Set a key with metadata in the backend servicing an application id
$meta = { 'rack_id' => 183 }
$opts = { 'app_id' => 'myapp' }
simpkv::put("hosts/${facts['clientcert']}", $facts['ipaddress'], $meta, $opts)
Set a gobal key in the default backend
simpkv::put("hosts/${facts['clientcert']}", $facts['ipaddress'], { 'global' => true })
key
Data type: String[1]
The key to set. Must conform to the following:
-
Key must contain only the following characters:
- a-z
- 0-9
- The following special characters:
._:-/
-
Key may not contain '/./' or '/../' sequences.
value
Data type: NotUndef
The value of the key
metadata
Data type: Optional[Hash]
Additional information to be persisted
options
Data type: Optional[Hash]
simpkv configuration that will be merged with
simpkv::options
. All keys are optional.
Options:
-
'app_id'
String
: Specifies an application name that can be used to identify which backend configuration to use via fuzzy name matching, in the absence of thebackend
option.- More flexible option than
backend
. - Useful for grouping together simpkv function calls found in different catalog resources.
- When specified and the
backend
option is absent, the backend will be selected preferring a backend in the mergedbackends
option whose name exactly matches theapp_id
, followed by the longest backend name that matches the beginning of theapp_id
, followed by thedefault
backend. - When absent and the
backend
option is also absent, this function will use thedefault
backend.
- More flexible option than
-
'backend'
String
: Definitive name of the backend to use.- Takes precedence over
app_id
. - When present, must match a key in the
backends
option of the merged options Hash or the function will fail. - When absent in the merged options, this function will select
the backend as described in the
app_id
option.
- Takes precedence over
-
'backends'
Hash
: Hash of backend configurations-
Each backend configuration in the merged options Hash must be a Hash that has the following keys:
type
: Backend type.id
: Unique name for the instance of the backend. (Same backend type can be configured differently).
-
Other keys for configuration specific to the backend may also be present.
-
-
'global'
Boolean
: Set totrue
when the key being accessed is global. Otherwise, the key will be tied to the Puppet environment of the node whose manifest is being compiled.- Defaults to
false
- Defaults to
-
'softfail'
Boolean
: Whether to ignore simpkv operation failures.- When
true
, this function will return a result even when the operation failed at the backend. - When
false
, this function will fail when the backend operation failed. - Defaults to
false
.
- When
simpkv::support::config::merge
Type: Ruby 4.x API
Create merged backend configuration and then validate it.
The merge entails the following operations:
-
The
options
argument is merged withsimpkv::options
Hiera and global simpkv defaults. -
If the
backend
options is missing in the merged options, it is set to a value determined as follows:- If the
app_id
option is present and the name of a backend inbackends
matchesapp_id
,backend
will be set toapp_id
. - Otherwise, if the
app_id
option is present and the name of a backend inbackends
matches the beginning ofapp_id
,backend
will be set to that partially-matching backend name. When multiple backends satisfy the 'start with' match, the backend with the most matching characters is selected. - Otherwise, if the
app_id
option does not match any backend name or is not present,backend
will be set todefault
.
- If the
-
If the
backends
option is missing in the merged options, it is set to a Hash containing a single entry,default
, that has configuration for the simpkv 'file' backend.
Validation includes the following checks:
- configuration for the selected backend exists
- the plugin for the selected backend has been loaded
- different configuration for a specific plugin instance does not exist
simpkv::support::config::merge(Hash $options, Array $backends)
Create merged backend configuration and then validate it.
The merge entails the following operations:
-
The
options
argument is merged withsimpkv::options
Hiera and global simpkv defaults. -
If the
backend
options is missing in the merged options, it is set to a value determined as follows:- If the
app_id
option is present and the name of a backend inbackends
matchesapp_id
,backend
will be set toapp_id
. - Otherwise, if the
app_id
option is present and the name of a backend inbackends
matches the beginning ofapp_id
,backend
will be set to that partially-matching backend name. When multiple backends satisfy the 'start with' match, the backend with the most matching characters is selected. - Otherwise, if the
app_id
option does not match any backend name or is not present,backend
will be set todefault
.
- If the
-
If the
backends
option is missing in the merged options, it is set to a Hash containing a single entry,default
, that has configuration for the simpkv 'file' backend.
Validation includes the following checks:
- configuration for the selected backend exists
- the plugin for the selected backend has been loaded
- different configuration for a specific plugin instance does not exist
Returns: Hash
merged simpkv options that will have the backend to use
specified by 'backend'
Raises:
ArgumentError
if merged configuration fails validation
options
Data type: Hash
Hash that specifies simpkv backend options to be merged with
simpkv::options
.
backends
Data type: Array
List of backends for which plugins have been successfully loaded.
simpkv::support::config::validate
Type: Ruby 4.x API
Validate backend configuration
simpkv::support::config::validate(Hash $options, Array $backends)
Validate backend configuration
Returns: Nil
Raises:
ArgumentError
if a backend has not been specified, appropriate configuration for a specified backend cannot be found, or different backend configurations are provided for the same ['type', 'id'] pair.
options
Data type: Hash
Hash that specifies simpkv backend options
backends
Data type: Array
List of backends for which plugins have been successfully loaded.
simpkv::support::key::validate
Type: Ruby 4.x API
Validates key conforms to the simpkv key specification
-
simpkv key specification
-
Key must contain only the following characters:
- a-z
- 0-9
- The following special characters:
._:-/
-
Key may not contain '/./' or '/../' sequences.
-
-
Terminates catalog compilation if validation fails.
Examples
Passing
simpkv::support::key::validate('looks/like/a/file/path')
simpkv::support::key::validate('looks/like/a/directory/path/')
simpkv::support::key::validate('simp-simp_snmpd:password.auth')
Failing
simpkv::support::key::validate('Uppercase/Characters/NOT/Allowed')
simpkv::support::key::validate('${special}/chars/not/allowed!'}
simpkv::support::key::validate('looks/like/an/./unexpanded/linux/path')
simpkv::support::key::validate('looks/like/another/../unexpanded/linux/path')
simpkv::support::key::validate(String[1] $key)
Validates key conforms to the simpkv key specification
-
simpkv key specification
-
Key must contain only the following characters:
- a-z
- 0-9
- The following special characters:
._:-/
-
Key may not contain '/./' or '/../' sequences.
-
-
Terminates catalog compilation if validation fails.
Returns: Nil
Raises:
ArgumentError
if validation fails
Examples
Passing
simpkv::support::key::validate('looks/like/a/file/path')
simpkv::support::key::validate('looks/like/a/directory/path/')
simpkv::support::key::validate('simp-simp_snmpd:password.auth')
Failing
simpkv::support::key::validate('Uppercase/Characters/NOT/Allowed')
simpkv::support::key::validate('${special}/chars/not/allowed!'}
simpkv::support::key::validate('looks/like/an/./unexpanded/linux/path')
simpkv::support::key::validate('looks/like/another/../unexpanded/linux/path')
key
Data type: String[1]
simpkv key
simpkv::support::load
Type: Ruby 4.x API
Load simpkv adapter and plugins and add simpkv 'extension' to the catalog instance, if it is not present
simpkv::support::load()
Load simpkv adapter and plugins and add simpkv 'extension' to the catalog instance, if it is not present
Returns: Nil
Raises:
LoadError
if simpkv adapter software fails to load
- Mon Oct 23 2023 Steven Pritchard steve@sicura.us - 0.13.0
- [puppetsync] Add EL9 support
- Wed Oct 11 2023 Steven Pritchard steve@sicura.us - 0.12.0
- [puppetsync] Updates for Puppet 8
- These updates may include the following:
- Update Gemfile
- Add support for Puppet 8
- Drop support for Puppet 6
- Update module dependencies
- These updates may include the following:
- Mon Aug 28 2023 Steven Pritchard steve@sicura.us - 0.11.0
- Add support for Puppet 8, Ruby 3, and stdlib 9
- Wed Aug 23 2023 Steven Pritchard steve@sicura.us - 0.10.0
- Add AlmaLinux 8 support
- Mon Jun 12 2023 Chris Tessmer chris.tessmer@onyxpoint.com - 0.9.0
- Add RockyLinux 8 support
- Tue Sep 28 2021 Liz Nemsick lnemsick.simp@gmail.com - 0.8.0
- BREAKING CHANGES:
- Added 'globals' and 'environments' root directories for global and
Puppet-environment keys, respectively, in the normalized key paths
in the backend.
- This change makes the top-level organization of keys in the backend explicit, and thus more understandable.
- The prefix used for global keys was changed from
<keystore root dir>
to<keystore root dir>/globals
. - The prefix used for environment keys was changed from
<keystore root dir>/<specific Puppet environment>
to<keystore root dir>/environments/<specific Puppet environment>
. - Change required for the LDAP plugin.
- Replaced the confusing 'environment' backend option in each simpkv Puppet
function with a 'global' Boolean option.
- Global keys are now specified by setting 'global' to true in lieu of setting 'environment' to ''.
- Changed the key and folder name specification to restrict letter
characters to lowercase.
- Change required for the LDAP plugin.
- Changed the plugin configuration API
- Configuration has been split out into its own method, instead of being done in the plugin constructor.
- This minimal change simplifies unit testing of configuration of complex plugins.
- Fixed the mechanism a plugin uses to advertise its type.
- Plugin type is now determined from its filename.
- Previous mechanism did not work when when multiple plugins were used.
- Added 'globals' and 'environments' root directories for global and
Puppet-environment keys, respectively, in the normalized key paths
in the backend.
- Added
- LDAP plugin
- Acceptance test that demonstrates its use and integration with a 389-DS instance configured with the SIMP data schema
- More detailed plugin exception reporting in order to pinpoint plugin
logic problems.
- Now prints out the useful portion of the backtrace when an exception is raised.
- Especially useful during plugin development.
- More background information for users
- More background information for plugin developers, which has now been split out into its own document.
- LDAP plugin
- Wed Jun 16 2021 Chris Tessmer chris.tessmer@onyxpoint.com - 0.8.0
- Removed support for Puppet 5
- Ensured support for Puppet 7 in requirements and stdlib
- Sat Dec 19 2020 Chris Tessmer chris.tessmer@onyxpoint.com - 0.7.2
- Removed EL6 support
- Fri Sep 18 2020 Liz Nemsick lnemsick.simp@gmail.com - 0.7.1
- Advertise EL8 support in metadata.json
- Fixed bad URL to docker's libkv project
- Wed Feb 26 2020 Trevor Vaughan tvaughan@onyxpoint.com - 0.7.0
- Changed name of module from libkv to simpkv
- Tue Oct 01 2019 Liz Nemsick lnemsick.simp@gmail.com - 0.7.0
- Merged in changes released with 0.6.1 and on the 'develop' branch
- Documented libkv requirements (See 'Requirements' section of docs/Design_Protoype2.md).
- Created and documented design for second prototype (See 'Changes from Version 0.6.X' section of docs/Design_Protoype2.md).
- Implemented second prototype and file plugin + store to demonstrate the design
- Wed Jan 31 2018 Dylan Cochran dylan.cochran@onyxpoint.com - 0.6.1
- Release 0.6.1
- Thu Oct 26 2017 Nick Markowski nicholas.markowski@onyxpoint.com - 0.6.0
- (SIMP-3923) Moved libkv::consul to pupmod-simp-simp_consul
- Updated README
- Updated travis with CI credentials
- Wed Oct 25 2017 Dylan Cochran dylan.cochran@onyxpoint.com - 0.5.0
- (SIMP-3445) Add acceptance test for consul configuration and bootstrap
- (SIMP-3629) libkv::atomic_put returns false for a successful put on consul >0.9.0
- Thu Aug 24 2017 Dylan Cochran dylan.cochran@onyxpoint.com - 0.4.3
- (SIMP-3623) libkv::lookup_key backend should turn softfail on by default
- Add fix for 0.9.x consul installations
- Tue Jul 18 2017 Dylan Cochran dylan.cochran@onyxpoint.com - 0.4.2
- (SIMP-3001) Prevent '.' and '..' from being used in keys
- (SIMP-3446) Add parameters to reconfigure http and https listen
- (SIMP-3429) libkv::list isn't always removing keys
- Tue Jul 18 2017 Dylan Cochran dylan.cochran@onyxpoint.com - 0.4.1
- Always copy over consul-acl, and update metadata
- Tue Jul 18 2017 Dylan Cochran dylan.cochran@onyxpoint.com - 0.4.0
- (SIMP-3275) libkv auto-config uses the root acl
- Tue Jul 11 2017 Dylan Cochran dylan.cochran@onyxpoint.com - 0.3.3
- (SIMP-3406) Fix docker containers for travisci
- (SIMP-3128) Delete .meta keys
- Tue Jul 11 2017 Dylan Cochran dylan.cochran@onyxpoint.com - 0.3.2
- (SIMP-3407) Fix idempoency on acl token generation
- (SIMP-3403) Spurious 'undefined method unpack'
- Mon Jul 10 2017 Dylan Cochran dylan.cochran@onyxpoint.com - 0.3.1
- (SIMP-3360) Use module data for certificate paths
- (SIMP-3087) Add libkv::lookup hierav5 backend function
- Mon Jul 10 2017 Dylan Cochran dylan.cochran@onyxpoint.com - 0.3.0
- (SIMP-2961) Add automatic cluster creation for consul.
- (SIMP-3130) metadata needs to default to 'String'.
- (SIMP-3129) atomic_create needs to create metadata
- (SIMP-3127) libkv can't list / since metadata update
- (SIMP-3122) Move the libkv wrapper outside the loader
- (SIMP-3125) Move key regex match into libkv wrapper
- (SIMP-3110) Use .meta to convert a value to the correct type
- (SIMP-3060) Fix travisci tests
- (SIMP-3109) Create a .meta key to store type
- Sat Apr 29 2017 Dylan Cochran dylan.cochran@onyxpoint.com - 0.2.0
- (SIMP-2978) Fix readme generation
- (SIMP-3019) Add ssl/tls support to consul backend
- (SIMP-2962) Add consul_members fact
- (SIMP-3097) Add 'serialize' and 'mode' parameters to all libkv functions
- (SIMP-3102) Flesh out spec tests
- (SIMP-2964) Add generic 'libkv::auth' parameter
- Fri Jan 6 2017 Dylan Cochran dylan.cochran@onyxpoint.com - 0.1.0
- Initial release
Dependencies
- puppetlabs/stdlib (>= 8.0.0 < 10.0.0)
simpkv - Per Section 105 of the Copyright Act of 1976, these works are not entitled to domestic copyright protection under US Federal law. The US Government retains the right to pursue copyright protections outside of the United States. The United States Government has unlimited rights in this software and all derivatives thereof, pursuant to the contracts under which it was developed and the License under which it falls. --- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.