Version information
This version is compatible with:
- Puppet Enterprise 2023.2.x, 2023.1.x, 2023.0.x, 2021.7.x, 2021.6.x, 2021.5.x, 2021.4.x, 2021.3.x, 2021.2.x, 2021.1.x, 2021.0.x, 2019.8.x, 2019.7.x, 2019.5.x, 2019.4.x, 2019.3.x, 2019.2.x, 2019.1.x, 2019.0.x, 2018.1.x, 2017.3.x, 2017.2.x, 2016.4.x
- Puppet >= 4.10.0 < 8.0.0
- CentOS, OracleLinux, RedHat, Scientific, Debian, Ubuntu, SLES, Fedora
Start using this module
Add this module to your Puppetfile:
mod 'southalc-vault_secrets', '0.2.1'
Learn more about managing modules with a PuppetfileDocumentation
vault_secrets
Table of Contents
Description
This module provides functions to integrate Hashicorp Vault with Puppet and uses the main module class to enable issuing and renewing host certificates from a Vault PKI secrets engine. The module has no dependencies other than 'puppetlabs-stdlib' and requires no extra ruby gems on the Puppet server or agents. If you only want to use the functions without issuing host certificates from Vault, add the module to an environment without assigning it to any nodes.
The custom hiera backend vault_hiera_hash
enables the puppet server to use a Vault key/value secrets engine as
a hiera data_hash. This allows storing multiple key/value pairs on a given Vault path, then using it from hiera
much like local YAML files. The function supports both v1 and v2 of the Vault key/value secrets engine and is
useful to easily separate secrets from other version controlled hiera data.
The vault_hash
and vault_key
functions also support Vault key/value secrets engines version 1 and 2, and can
be used in manifests to get secrets from Vault. With Puppet agents 6 and later you can also use
deferred functions to enable clients to get
secrets directly from Vault without passing them through the Puppet server.
Setup
Obviously you need to have a working Hashicorp Vault deployment first. Complete the following steps on the Vault server:
- Enable a kv secrets engine on path 'puppet/'. This example is for version 1 of the k/v engine:
vault secrets enable -path=puppet -description="Puppet data" -version=1 kv
-
Use the Vault UI or command line to create a secret under the 'puppet/' path. This example will use the name "common" as the secret name for storing key/value secrets common to all nodes.
-
Configure Vault to enable authentication with certificates issued by the Puppet certificate authority. From a Puppet node you can reference the Puppet CA certificate from: '/etc/puppetlabs/puppet/ssl/certs/ca.pem'. When using PKI authentication, the module functions will authenticate every Vault request independently, so use a low "ttl" value to keep from building up many active Vault tokens. Alternatively, you can generate a long-term Vault token for use with the
vault_hiera_hash
function as a way to reduce overhead during hiera lookups.
vault auth enable -path=puppet-pki -description="PKI authentication with Puppet certificates" cert
vault write auth/puppet-pki/certs/puppetCA \
display_name="Puppet CA" \
policies=puppet \
certificate=@/etc/puppetlabs/puppet/ssl/certs/ca.pem \
ttl=60
If you don't want to use deferred functions and just want to use Vault with hiera, it's recommended to manually create a Vault token. This avoids the overhead of certificate authentication with every hiera lookup. Set up a Vault policy for Puppet first per the example in setup step #5, then create the Vault token:
vault token create -display-name="Puppet hiera lookups" -orphan -policy=puppet
- Optionally enable a PKI secrets engine on Vault to serve as a certificate authority and get dynamic provisioning of X.509 certificates.
vault secrets enable -description="PKI service" -path=pki pki
With the 'pki' secrets engine enabled, create a certificate authority for Vault. There are several options to set this up, depending on your security requirements. Choose one of the following methods:
Option 1 - Create a new root CA in Vault. When setting up a root CA, you will likely want to increase the max lifetime of the certificate first. It seems Vault doesn't understand logical units for days or years, so we are forced in to using hours (this example is ~10 years in hours):
vault secrets tune -max-lease-ttl=87600h pki
With the max lifetime set to a reasonable value for a root CA, generate the new root certificate in Vault, using the max TTL:
vault write pki/root/generate/internal common_name='Vault Root CA' ttl=87600h
Option 2 - From Vault, create a certificate signing request for a subordinate certificate authority. Submit the CSR to an external CA for signing, then import the signed certificate into Vault. First, set the max lifetime of certificates to a good working value for a subordinate CA. This example is 5 years, in hours:
vault secrets tune -max-lease-ttl=43800h pki
Generate the subordinate CSR from Vault, using a common name as you like and using the max TTL we just defined:
vault write pki/intermediate/generate/internal common_name="Sub CA" ttl=43800h
Capture the CSR from the output of the last command and submit it to the upstream CA for signing. When you get the signed certificate back, import the upstream CA certificate into Vault, then import the the signed subordinate CA certificate:
vault write pki/intermediate/set-signed certificate=@signed_certificate.pem
Option 3 - Externally from Vault, generate a signed certificate for a subordinate CA. Create a certificate chain beginning with the private key of the subordinate CA, the signed subordinate certificate, any intermediate CA certificates, and finally the root CA certificate. The private key of the new subordinate CA will need to be in decrypted form for the certificate chain to be imported into to Vault:
vault write pki/config/ca pem_bundle=@subca-chain.pem
With the certificate authority established, define the endpoints for issuing certificates and revocation lists. These can be modified later if needed:
vault write pki/config/urls \
issuing_certificates="http://vault.example.local:8200/v1/pki/ca" \
crl_distribution_points="http://vault.example.local:8200/v1/pki/crl"
Configure one or more roles for the pki service. The role is where the certificate attributes can be defined. Policy can be used to enable access to different roles as needed:
vault write pki/roles/example-local \
allowed_domains=example.local \
allow_subdomains=true max_ttl=168h \
organization=Example \
ou=Automation \
country=US \
To view all the possible settings that could be defined by the role:
vault read pki/roles/example-local
To issue a certificate from the pki service, invoke a write to the role. We'll be using this Puppet module to automate this and to automatically renew certificates as they near the expiration date.
vault write pki/issue/example-local common_name=myhost.example.local
- Create a policy on Vault to enable access to Puppet data. This policy example grants read access to everything under the Vault path "puppet/nodes" and the specific Vault path "puppet/common". The "pki/issue/example-local" is granted 'update' access to enable Puppet to obtain PKI certificates from the configured role on the Vault certificate authority. Applying this policy to the 'puppet-pki' authentication path will enable any Puppet certificate to authenticate and access these Vault paths. Review the Vault documentation for managing access to Vault with policies. Save the policy to a file:
path "puppet/nodes/*" {
capabilities = ["read"]
}
path "puppet/common" {
capabilities = ["read"]
}
path "pki/issue/example-local" {
capabilities = ["update"]
}
Write the policy to Vault on the 'puppet-pki' authentication path:
vault policy write puppet-pki <file>
Usage
Here are some examples of using the module functions to obtain secrets from Vault. The 'vault_hash' function returns a hash of all keys/values from a Vault server on a given path, authenticating with the 'puppet-pki' method:
$vault_hash = vault_hash('https://vault.example.com:8200/v1/puppet/common', 'puppet-pki')
notify { 'hash_example' :
message => $vault_hash,
}
The 'vault_key' function returns only the value from the specified Vault URI, path, and key:
$vault_value = vault_key('https://vault.example.com:8200/v1/puppet/common', 'puppet-pki', 'secret_key')
notify { 'key_example' :
message => $vault_value,
}
Functions may be deferred so they run from the managed node instead of the Puppet server. This way the secret is passed directly from Vault to the client where it is needed:
$deferred_hash = Deferred('vault_hash', ['https://vault.example.com:8200/v1/puppet/common', 'puppet-pki'])
notify { 'deferred_example' :
message => $deferred_hash,
}
$deferred_value = Deferred('vault_key', ['https://vault.example.com:8200/v1/puppet/common', 'puppet-pki', 'secret_key'])
notify { 'deferred_example' :
message => $deferred_value,
}
Vault as a hiera backend
Configure an environment to use the custom hiera backend by using the 'vault_hiera_hash' function for the data_hash. This enables the Puppet server to lookup all key/value pairs stored on a Vault secrets path in a single lookup. You can also pass an array of 'uris' that may include variable references to be interpolated at runtime. Note that each uri results in a separate Vault request and will incur some overhead on the Puppet server. The 'options' hash requires a 'ca_trust' value, which is the certificate authority chain used to validate the TLS connection to Vault.
hierarchy:
- name: "Secrets from Vault"
data_hash: vault_hiera_hash
uri: "https://vault.example.com:8200/v1/puppet/common" # Secrets common to all nodes
options:
timeout: 3
auth_path: "puppet-pki"
ca_trust: "/etc/ssl/certs/ca-certificates.crt"
Example configuration using Vault for hiera data using multiple search paths and a pre-staged Vault token to avoid authentication overhead with each lookup. When using a Vault token, ensure the 'token_file' has read permissions by the user running the Puppet server process. Obviously this is a security sensitive file that should be protected from all other access.
hierarchy:
- name: "Secrets from Vault"
data_hash: vault_hiera_hash
uris:
- "https://vault.example.com:8200/v1/puppet/nodes/%{fqdn}" # Node specific secrets
- "https://vault.example.com:8200/v1/puppet/common" # Secrets common to all nodes
options:
timeout: 3
token_file: "/etc/puppetlabs/puppet/.vault-token" # File contains the Vault token
ca_trust: "/etc/ssl/certs/ca-certificates.crt"
Limitations
Should work with all supported releases of Puppet server, but has been only minimally tested. Deferred functinos require Puppet 6 or later.
Reference
Table of Contents
Classes
vault_secrets
: Issue and renew PKI certificates from Hashicorp Vault
Functions
vault_cert
: Obtain a host certificate from a Vault PKI secrets enginevault_hash
: Return a hash from a Vault key/value secrets engine pathvault_hiera_hash
: Custom hiera back-end for Hashicorp Vault key/value secrets enginevault_key
: Return the value from a Vault key/value secrets engine from a given path and key
Classes
vault_secrets
Issue and renew PKI certificates from Hashicorp Vault
Examples
Issue a host certificate from a Vault server with PKI secrets engine
class { 'vault_secrets':
vault_uri => 'https://vault.example.com:8200/v1/pki/issue/example-com',
auth_path => 'puppet-pki',
}
Parameters
The following parameters are available in the vault_secrets
class.
vault_uri
Data type: String
The complete URL of the the Hashicorp Vault certificate issuing role API endpoint
auth_path
Data type: String
The Vault path of the authentication provider used by Puppet certificates
days_before_renewal
Data type: Integer[1, 30]
The number of days before expiration where the host certificate will be re-issued
Default value: 3
cert_data
Data type: Hash
A hash of values to be submitted with the certificate request. The hash contents should adhere to the keys/values supported/permitted by the PKI role and policy. Basic default values are defined in module hiera.
Default value: {}
Functions
vault_cert
Type: Ruby 4.x API
Obtain a host certificate from a Vault PKI secrets engine
vault_cert(String $vault_uri, String $auth_path, Hash $data, Optional[Integer] $timeout)
The vault_cert function.
Returns: Hash
The returned hash contains the certificate, private key, and supporting data
vault_uri
Data type: String
The complete API path to a Vault PKI role for issuing certificates.
auth_path
Data type: String
The Vault mount path of the "cert" authentication type used with Puppet certificates.
data
Data type: Hash
A hash of values to be submitted with a certificate request. The hash contents must adhere to the constructs of the Vault PKI role and policy being used at the 'vault_uri' endpoint.
timeout
Data type: Optional[Integer]
Value in seconds to wait for Vault connections. Default is 5.
vault_hash
Type: Ruby 4.x API
Return a hash from a Vault key/value secrets engine path
vault_hash(String $vault_uri, String $auth_path, Optional[String] $version, Optional[Integer] $timeout)
The vault_hash function.
Returns: Hash
Contains all the key/value pairs from the given path.
vault_uri
Data type: String
The full URI to the Vault API endpoint for a key/value secrets engine path.
auth_path
Data type: String
The Vault mount path of the 'cert' authentication type used with Puppet certificates.
version
Data type: Optional[String]
Set this value to 'v2' to use version 2 of the Vault key/value secrets engine.
timeout
Data type: Optional[Integer]
Value in seconds to wait for Vault connections.
vault_hiera_hash
Type: Ruby 4.x API
Custom hiera back-end for Hashicorp Vault key/value secrets engine
vault_hiera_hash(Hash $options, Puppet::LookupContext $context)
The vault_hiera_hash function.
Returns: Hash
All key/value pairs from the given Vault path will be returned to hiera
options
Data type: Hash
Hash containing:
context
Data type: Puppet::LookupContext
Default parameter used for caching
vault_key
Type: Ruby 4.x API
Return the value from a Vault key/value secrets engine from a given path and key
vault_key(String $vault_uri, String $auth_path, String $key, Optional[String] $version, Optional[Integer] $timeout)
The vault_key function.
Returns: String
The value of the secret from the @vault_uri and @key
vault_uri
Data type: String
The complete API path to a Vault key/value secret
auth_path
Data type: String
The Vault mount path of the "cert" authentication type used with Puppet certificates.
key
Data type: String
The name of a specific secret at the given 'vault_uri'
version
Data type: Optional[String]
Set this value to 'v2' to use version 2 of the Vault key/value secrets engine
timeout
Data type: Optional[Integer]
Value in seconds to wait for a response from Vault
Changelog
Release 0.2.1
Bugfix - Module hiera update for new module name
Release 0.2.0
- Module renamed to "vault_secrets" to avoid naming conflict
- Updated error handling in functions
- vault_hiera_hash function now uses 'token_file' instead of 'token'
Release 0.1.1
Bugfix - Set 'show_diff' to false when updating certificate and private key file content
Release 0.1.0
Initial release - See README for features
Dependencies
- puppetlabs/stdlib (>= 4.19.0 < 8.0.0)