Forge Home

Modules

Resources

PuppetEcosystemPuppet CommunityUpdates

vault_secrets

Work with secrets from Hashicorp Vault
Project URLRSS FeedReport issues

5,058 downloads

217 latest version

5.0 quality score

We run a couple of automated
scans to help you access a
module's quality. Each module is
given a score based on how well
the author has formatted their
code and documentation and
modules are also checked for
malware using VirusTotal.

Please note, the information below
is for guidance only and neither of
these methods should be considered
an endorsement by Puppet.

Version information

  • 0.6.1 (latest)
  • 0.6.0
  • 0.5.2
  • 0.5.1
  • 0.5.0
  • 0.4.5
  • 0.4.4
  • 0.4.3
  • 0.4.2
  • 0.4.1
  • 0.4.0
  • 0.3.2
  • 0.3.1
  • 0.3.0
  • 0.2.1
  • 0.2.0 (deleted)
released Oct 16th 2021
This version is compatible with:
  • Puppet Enterprise 2023.2.x, 2023.1.x, 2023.0.x, 2021.7.x, 2021.6.x, 2021.5.x, 2021.4.x, 2021.3.x, 2021.2.x, 2021.1.x, 2021.0.x, 2019.8.x, 2019.7.x, 2019.5.x, 2019.4.x, 2019.3.x, 2019.2.x, 2019.1.x, 2019.0.x, 2018.1.x, 2017.3.x, 2017.2.x, 2016.4.x
  • Puppet >= 4.10.0 < 8.0.0
  • CentOS, OracleLinux, RedHat, Scientific, Debian, Ubuntu, SLES, Fedora

Start using this module

  • r10k or Code Manager
  • Bolt
  • Manual installation
  • Direct download

Add this module to your Puppetfile:

mod 'southalc-vault_secrets', '0.3.2'
Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add southalc-vault_secrets
Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install southalc-vault_secrets --version 0.3.2

Direct download is not typically how you would use a Puppet module to manage your infrastructure, but you may want to download the module in order to inspect the code.

Download
Tags: certificate, hiera, secrets, vault, cert

Documentation

southalc/vault_secrets — version 0.3.2 Oct 16th 2021

vault_secrets

Table of Contents

  1. Description
  2. Vault Setup
  3. Usage
  4. Reference
  5. Limitations
  6. Development

Description

This module provides functions and type/providers to integrate Hashicorp Vault with Puppet and uses the main module class to enable issuing and renewing host certificates from a Vault PKI secrets engine. The module has no dependencies other than 'puppetlabs-stdlib' and requires no extra ruby gems on the Puppet server or agents. If you only want to use the functions without issuing host certificates from Vault, add the module to an environment without assigning it to any nodes.

The custom hiera backend vault_hiera_hash enables the puppet server to use a Vault key/value secrets engine as a hiera data_hash. This allows storing multiple key/value pairs on a given Vault path, then using it from hiera much like local YAML files. The function supports both v1 and v2 of the Vault key/value secrets engine and is useful to easily separate secrets from other version controlled hiera data.

The vault_hash and vault_key functions also support Vault key/value secrets engines version 1 and 2, and can be used in manifests to get secrets from Vault. With Puppet agents 6 and later you can also use deferred functions to enable clients to get secrets directly from Vault without passing them through the Puppet server.

The vault_cert resource type can be used to manage TLS certificates for use by other applications. These are issued by Vault directly to agents (private keys are not stored on puppet servers or in the catalog), and renewed automatically as needed.

Setup

Obviously you need to have a working Hashicorp Vault deployment first. Complete the following steps on the Vault server:

  1. Enable a kv secrets engine on path 'puppet/'. This example is for version 1 of the k/v engine:
vault secrets enable -path=puppet -description="Puppet data" -version=1 kv

  1. Use the Vault UI or command line to create a secret under the 'puppet/' path. This example will use the name "common" as the secret name for storing key/value secrets common to all nodes.

  2. Configure Vault to enable authentication with certificates issued by the Puppet certificate authority. From a Puppet node you can reference the Puppet CA certificate from: '/etc/puppetlabs/puppet/ssl/certs/ca.pem'. When using PKI authentication, the module functions will authenticate every Vault request independently, so use a low "ttl" value to keep from building up many active Vault tokens. Alternatively, you can generate a long-term Vault token for use with the vault_hiera_hash function as a way to reduce overhead during hiera lookups.

vault auth enable -path=puppet-pki -description="PKI authentication with Puppet certificates" cert
vault write auth/puppet-pki/certs/puppetCA \
  display_name="Puppet CA" \
  policies=puppet \
  certificate=@/etc/puppetlabs/puppet/ssl/certs/ca.pem \
  ttl=60

If you don't want to use deferred functions and just want to use Vault with hiera, it's recommended to manually create a Vault token. This avoids the overhead of certificate authentication with every hiera lookup. Set up a Vault policy for Puppet first per the example in setup step #5, then create the Vault token:

vault token create -display-name="Puppet hiera lookups" -orphan -policy=puppet
  1. Optionally enable a PKI secrets engine on Vault to serve as a certificate authority and get dynamic provisioning of X.509 certificates.
vault secrets enable -description="PKI service" -path=pki pki

With the 'pki' secrets engine enabled, create a certificate authority for Vault. There are several options to set this up, depending on your security requirements. Choose one of the following methods:

Option 1 - Create a new root CA in Vault. When setting up a root CA, you will likely want to increase the max lifetime of the certificate first. It seems Vault doesn't understand logical units for days or years, so we are forced in to using hours (this example is ~10 years in hours):

vault secrets tune -max-lease-ttl=87600h pki

With the max lifetime set to a reasonable value for a root CA, generate the new root certificate in Vault, using the max TTL:

vault write pki/root/generate/internal common_name='Vault Root CA'  ttl=87600h

Option 2 - From Vault, create a certificate signing request for a subordinate certificate authority. Submit the CSR to an external CA for signing, then import the signed certificate into Vault. First, set the max lifetime of certificates to a good working value for a subordinate CA. This example is 5 years, in hours:

vault secrets tune -max-lease-ttl=43800h pki

Generate the subordinate CSR from Vault, using a common name as you like and using the max TTL we just defined:

vault write pki/intermediate/generate/internal common_name="Sub CA" ttl=43800h

Capture the CSR from the output of the last command and submit it to the upstream CA for signing. When you get the signed certificate back, import the upstream CA certificate into Vault, then import the the signed subordinate CA certificate:

vault write pki/intermediate/set-signed certificate=@signed_certificate.pem

Option 3 - Externally from Vault, generate a signed certificate for a subordinate CA. Create a certificate chain beginning with the private key of the subordinate CA, the signed subordinate certificate, any intermediate CA certificates, and finally the root CA certificate. The private key of the new subordinate CA will need to be in decrypted form for the certificate chain to be imported into to Vault:

vault write pki/config/ca pem_bundle=@subca-chain.pem

With the certificate authority established, define the endpoints for issuing certificates and revocation lists. These can be modified later if needed:

vault write pki/config/urls \
  issuing_certificates="http://vault.example.local:8200/v1/pki/ca" \
  crl_distribution_points="http://vault.example.local:8200/v1/pki/crl"

Configure one or more roles for the pki service. The role is where the certificate attributes can be defined. Policy can be used to enable access to different roles as needed:

vault write pki/roles/example-local \
  allowed_domains=example.local \
  allow_subdomains=true max_ttl=168h \
  organization=Example \
  ou=Automation \
  country=US \

To view all the possible settings that could be defined by the role:

vault read  pki/roles/example-local

To issue a certificate from the pki service, invoke a write to the role. We'll be using this Puppet module to automate this and to automatically renew certificates as they near the expiration date.

vault write pki/issue/example-local common_name=myhost.example.local
  1. Create a policy on Vault to enable access to Puppet data. This policy example grants read access to everything under the Vault path "puppet/nodes" and the specific Vault path "puppet/common". The "pki/issue/example-local" is granted 'update' access to enable Puppet to obtain PKI certificates from the configured role on the Vault certificate authority. Applying this policy to the 'puppet-pki' authentication path will enable any Puppet certificate to authenticate and access these Vault paths. Review the Vault documentation for managing access to Vault with policies. Save the policy to a file:
path "puppet/nodes/*" {
  capabilities = ["read"]
}
path "puppet/common" {
  capabilities = ["read"]
}
path "pki/issue/example-local" {
  capabilities = ["update"]
}

Write the policy to Vault on the 'puppet-pki' authentication path:

vault policy write puppet-pki <file>

Usage

Here are some examples of using the module functions to obtain secrets from Vault. The 'vault_hash' function returns a hash of all keys/values from a Vault server on a given path, authenticating with the 'puppet-pki' method:

$vault_hash = vault_hash('https://vault.example.com:8200/v1/puppet/common', 'puppet-pki')
notify { 'hash_example' :
  message => $vault_hash,
}

The 'vault_key' function returns only the value from the specified Vault URI, path, and key:

$vault_value = vault_key('https://vault.example.com:8200/v1/puppet/common', 'puppet-pki', 'secret_key')
notify { 'key_example' :
  message => $vault_value,
}

Functions may be deferred so they run from the managed node instead of the Puppet server. This way the secret is passed directly from Vault to the client where it is needed:

$deferred_hash = Deferred('vault_hash', ['https://vault.example.com:8200/v1/puppet/common', 'puppet-pki'])
notify { 'deferred_example' :
  message => $deferred_hash,
}

$deferred_value = Deferred('vault_key', ['https://vault.example.com:8200/v1/puppet/common', 'puppet-pki', 'secret_key'])
notify { 'deferred_example' :
  message => $deferred_value,
}

Vault as a hiera backend

Configure an environment to use the custom hiera backend by using the 'vault_hiera_hash' function for the data_hash. This enables the Puppet server to lookup all key/value pairs stored on a Vault secrets path in a single lookup. You can also pass an array of 'uris' that may include variable references to be interpolated at runtime. Note that each uri results in a separate Vault request and will incur some overhead on the Puppet server. The 'options' hash requires a 'ca_trust' value, which is the certificate authority chain used to validate the TLS connection to Vault.

hierarchy:
  - name: "Secrets from Vault"
    data_hash: vault_hiera_hash
    uri: "https://vault.example.com:8200/v1/puppet/common"   # Secrets common to all nodes
    options:
      timeout: 3
      auth_path: "puppet-pki"
      ca_trust: "/etc/ssl/certs/ca-certificates.crt"

Example configuration using Vault for hiera data using multiple search paths and a pre-staged Vault token to avoid authentication overhead with each lookup. When using a Vault token, ensure the 'token_file' has read permissions by the user running the Puppet server process. Obviously this is a security sensitive file that should be protected from all other access. A Vault agent with an app role is useful for this.

hierarchy:
  - name: "Secrets from Vault"
    data_hash: vault_hiera_hash
    uris:
      - "https://vault.example.com:8200/v1/puppet/nodes/%{fqdn}"  # Node specific secrets
      - "https://vault.example.com:8200/v1/puppet/common"         # Secrets common to all nodes
    options:
      timeout: 3
      token_file: "/run/puppet/vault-token"                       # File contains the Vault token
      ca_trust: "/etc/ssl/certs/ca-certificates.crt"

Vault-issued certificates directly to nodes

Include the vault_secrets::vault_cert class to ensure required directories are created and then use the vault_cert resource type for each certificate you wish to manage. This resource will authenticate to vault using the agent's Puppet certificate, and write the key, certificate and chain files so they can be consumed by another application. The private key is written only on the agent, and is not revealed to the puppet server or included in the catalog.

Certificates will be renewed automatically when puppet runs and there are fewer days remaining than the renewal_threshold parameter is set to, or if the certificate is found to already be expired.

By default, all files are written to the directory specified by the $::vault_cert_dir fact, however the issued certificate and key files can be written to arbitrary locations. File ownership and permissions can also be customised as shown below.

include vault_secrets::vault_cert

vault_cert { 'test':
  ensure            => present,
  vault_uri         => 'https://vault.example.com:8200/pki/issue/role',
  cert_data         => {
    'common_name' => 'test.example.com',
    'alt_names'   => ['alias.exmaple.com', 'localhost'].join('\n'),
    'ip_sans'     => [$::facts['networking']['ip'], '127.0.0.1'],
    'ttl'         => '2160h', # 90 days
  },
  # Optional
  renewal_threshold => 5,
  ca_chain_file     => '/srv/myapp/ca.crt',
  ca_chain_owner    => 'myapp',
  ca_chain_group    => 'myapp',
  cert_file         => '/srv/myapp/server.crt',
  cert_owner        => 'myapp',
  cert_group        => 'myapp',
  key_file          => '/srv/myapp/server.key',
  key_owner         => 'myapp',
  key_group         => 'myapp',
}

You can purge certificates from the system which are no longer included in the puppet catalog by setting vault_sercrets::vault_cert::purge: true in hiera, or like the below:

class { 'vault_secrets::vault_cert':
  purge => true,
}

vault_cert resources will autorequire any File resources coresponding to the parent directories of the ca_chain_file, cert_file and key_file properties. Any User or Group resources corresponding to the *_owner or *_group properties will also be autorequired.

Limitations

Should work with all supported releases of Puppet server, but has been only minimally tested. Deferred functinos require Puppet 6 or later.