Puppet Enterprise (pe) Administration (adm) Module

This Puppet module contains Bolt plans used to deploy and manage Puppet Enterprise infrastructure. Plans are provided to automate common lifecycle activities in order to increase velocity and reduce the possibility of human error incurred by manually performing these activities.

The peadm module is able to deploy and manage Puppet Enterprise 2019.x Standard, Large, and Extra Large architectures.

Table of Contents

1. Expectations and support
2. Overview
   • What peadm affects
   • What peadm does not affect
   • Requirements
3. Usage
4. Reference
5. Getting Help

Expectations and support

The peadm module is intended to be used only by Puppet Enterprise customers actively working with and being guided by Puppet Customer Success teams—specifically, the Professional Services and Solutions Architecture teams. Puppet Enterprise customers might be advised to begin using this tool after an introduction by their Technical Account manager (TAM) to the Solutions Architecture team or during a Professional Sevices (PS) engagement where the Puppet PS team aids and intructurs in the use of the tool. Independent use is not recommended for production environments without a comprehensive understanding of the peadm module.

The peadm module is a services-led tool that is supported through Puppet Enterprise's standard and premium support.puppet.com service.

Overview

The normal usage pattern for peadm is as follows.

1. Users set up a Bolt host from which they can run peadm plans. The Bolt host can be any machine that has ssh access to all of the PE nodes.
2. Users run the peadm::install plan to bootstrap a new PE cluster. Depending on the architecture chosen, peadm may create some node groups in the classifier to set parameters on the built-in puppet_enterprise module, tuning it for large or extra large architectures.
3. Users use and operate their PE cluster as normal. The peadm module is not used again until the next upgrade.
4. When it is time to upgrade, users run the peadm::upgrade plan from their Bolt host to accelerate and aid in the upgrade process.

What peadm affects

• The peadm::install plan adds a number of custom OID trusted facts to the certificates of PE infrastructure nodes as it deploys them. These trusted facts are later used by the plans to quickly and correctly identify nodes in particular roles.
• Up to four node groups may be created to help configure puppet_enterprise class parameters for PE infrastructure roles. The most notable configuration is the designation of compilers as being either "A" or "B" nodes for availability.

What peadm does not affect

• The peadm module is not required to exist or be present outside of the point(s) in time it is used to create a new PE cluster, or upgrade an existing cluster. No new Puppet classes or other persistent content not provided out-of-box by PE itself is applied to PE infrastructure nodes by the peadm module.
• Having used the peadm module to install or to upgrade a PE cluster is not known to affect or curtail the ability to use any normal, documented PE procedures, e.g. failover to a replica, or manual upgrade of a cluster.

Requirements

• Puppet Enterprise 2019.8.1 or newer (tested with PE 2021.6)
• Bolt 3.17.0 or newer (tested with Bolt 3.21.0)
• EL 7, EL 8, Ubuntu 18.04, or Ubuntu 20.04
• Classifier Data enabled. This PE feature is enabled by default on new installs, but can be disabled by users if they remove the relevant configuration from their global hiera.yaml file. See the PE docs for more information.

Usage

Follow the links below to usage instructions for each peadm plan.

• Install
• Upgrade
• Convert
• Status

Reference

Additional documentation and information pertaining to various aspects or elements of peadm.

• DR Component Recovery
• PE Architecture Documentation
• Classification
• Architectures
• Testing
• Docker Based Examples

Getting Help

• If you find bugs with this module, please make use of issues in the project on GitHub
• If you are a Puppet Enterprise (PE) customer that uses peadm to manage a deployment of PE and are currently having an outage or need assistance troubleshooting another issue, e.g. upgrades, contact the Support Team

Reference

Table of Contents

Classes

Public Classes

Private Classes

• peadm::setup::convert_node_manager: Used during the peadm::convert plan
• peadm::setup::convert_pre20197: Defines configuration needed for converting PE 2018
• peadm::setup::node_manager: Configures PEAdm's required node groups
• peadm::setup::node_manager_yaml: Set up the node_manager.yaml file in the temporary Bolt confdir

Functions

• peadm::assert_supported_architecture: Assert that the architecture given is a supported one
• peadm::assert_supported_bolt_version: Assert that the Bolt executable running PEAdm is a supported version
• peadm::assert_supported_pe_version: Assert that the PE version given is supported by PEAdm
• peadm::bolt_version
• peadm::certname: Return the certname of the given target-like input
• peadm::convert_hash: converts two arrays into hash
• peadm::convert_status: Transforms a value in a human readable status with or without colors
• peadm::determine_status: Produces a summarized hash of the given status data
• peadm::fail_on_transport: Fails if any nodes have the chosen transport. Useful for excluding PCP when it's not appopriate
• peadm::file_content_upload
• peadm::file_or_content
• peadm::flatten_compact
• peadm::generate_pe_conf: Generate a pe.conf file in JSON format
• peadm::get_targets: Accept undef or a SingleTargetSpec, and return an Array[Target, 1, 0]. This differs from get_target() in that: - It returns an Array[Target
• peadm::node_manager_yaml_location
• peadm::oid
• peadm::plan_step
• peadm::recovery_opts_default
• peadm::wait_until_service_ready: A convenience function to help remember port numbers for services and handle running the wait_until_service_ready task

Data types

• Peadm::Ldap_config
• Peadm::Pe_version
• Peadm::Pem
• Peadm::Recovery_opts
• Peadm::SingleTargetSpec: A SingleTargetSpec represents any String, Target or single-element array of one or the other that can be passed to get_targets() to return an

Tasks

• agent_install: Install the Puppet agent from a master
• agent_upgrade: Upgrade the target system using upgrade.bash from a master
• backup_classification: A task to call the classification api and write to file
• cert_data: Return certificate data related to the Puppet agent
• cert_valid_status: Check primary for valid state of a certificate
• code_manager: Perform various code manager actions
• code_sync_status: A task to confirm code is in sync accross the cluster for clusters with code manager configured
• divert_code_manager: Divert the code manager live-dir setting
• download: Download a file using curl
• enable_replica: Execute the enable replica puppet command
• filesize: Return the size of a file in bytes
• get_peadm_config: Run on a PE primary node to return the currently configured PEAdm parameters
• infrastatus: Runs puppet infra status and returns the output
• mkdir_p_file: Create a file with the specified content at the specified location
• mv: Wrapper task for mv command
• pe_install: Install Puppet Enterprise from a tarball
• pe_ldap_config: Set the ldap config in the PE console
• pe_uninstall: Uninstall Puppet Enterprise
• precheck: Return pre-check information about a system
• provision_replica: Execute the replica provision puppet command
• puppet_infra_upgrade: Execute the puppet infra upgrade command
• puppet_runonce: Run the Puppet agent one time
• rbac_token: Get and save an rbac token for the root user, admin rbac user
• read_file: Read the contents of a file
• restore_classification: A short description of this task
• sign_csr: Submit a certificate signing request
• ssl_clean: Clean an agent's certificate
• submit_csr: Submit a certificate signing request
• transform_classification_groups: Transform the user groups from a source backup to a list of groups on the target server
• wait_until_service_ready: Return when the orchestrator service is healthy, or timeout after 15 seconds

Plans

Public Plans

• peadm::add_database
• peadm::convert: Convert an existing PE cluster to a PEAdm-managed cluster
• peadm::install: Install a new PE cluster
• peadm::modify_certificate: Modify the certificate of one or more targets
• peadm::status: Return status information from one or more PE clusters in a table format
• peadm::upgrade: Upgrade a PEAdm-managed cluster

Private Plans

• peadm::add_compiler: Add a new compiler to a PE architecture or replace an existing one with new configuration.
• peadm::add_replica: Replace a replica host for a Standard or Large architecture. Supported use cases: 1: The existing replica is broken, we have a fresh new VM we want to provision the replica to. The new replica should have the same certname as the broken one.
• peadm::backup: Backup the core user settings for puppet infrastructure
• peadm::misc::divert_code_manager: This plan exists to account for a scenario where a PE XL
• peadm::modify_cert_extensions
• peadm::restore: Restore the core user settings for puppet infrastructure from backup
• peadm::subplans::component_install: Install a new PEADM component
• peadm::subplans::configure: Configure first-time classification and DR setup
• peadm::subplans::db_populate: Destructively (re)populates a new or existing database with the contents or a known good source
• peadm::subplans::install: Perform initial installation of Puppet Enterprise Extra Large
• peadm::subplans::modify_certificate
• peadm::subplans::prepare_agent
• peadm::uninstall: Single-entry-point plan for uninstalling Puppet Enterprise
• peadm::util::code_sync_status
• peadm::util::db_disable_pglogical
• peadm::util::db_purge
• peadm::util::insert_csr_extension_requests
• peadm::util::retrieve_and_upload
• peadm::util::sanitize_pg_pe_conf
• peadm::util::sync_global_hiera
• peadm::util::update_classification: Configure classification
• peadm::util::update_db_setting: Make updates to PuppetDB database settings

Classes

Functions

peadm::assert_supported_architecture

Type: Puppet Language

Assert that the architecture given is a supported one

peadm::assert_supported_architecture(TargetSpec $primary_host, Variant[TargetSpec, Undef] $replica_host = undef, Variant[TargetSpec, Undef] $primary_postgresql_host = undef, Variant[TargetSpec, Undef] $replica_postgresql_host = undef, Variant[TargetSpec, Undef] $compiler_hosts = undef)

The peadm::assert_supported_architecture function.

Returns: Hash

primary_host

Data type: TargetSpec

replica_host

Data type: Variant[TargetSpec, Undef]

primary_postgresql_host

Data type: Variant[TargetSpec, Undef]

replica_postgresql_host

Data type: Variant[TargetSpec, Undef]

compiler_hosts

Data type: Variant[TargetSpec, Undef]

peadm::assert_supported_bolt_version

Type: Puppet Language

Checks if the current Bolt version matches the SemVerRange defined in $supported_bolt_version Fails the calling plan if false, does nothing if true. Accepts a parameter for the $supported_bolt_version for unit testing purposes

peadm::assert_supported_bolt_version()

Checks if the current Bolt version matches the SemVerRange defined in $supported_bolt_version Fails the calling plan if false, does nothing if true. Accepts a parameter for the $supported_bolt_version for unit testing purposes

Returns: Struct[{'supported' => Boolean}]

peadm::assert_supported_pe_version

Type: Puppet Language

Assert that the PE version given is supported by PEAdm

peadm::assert_supported_pe_version(String $version, Boolean $permit_unsafe_versions = false)

The peadm::assert_supported_pe_version function.

Returns: Struct[{'supported' => Boolean}] true if the version is supported, raise error otherwise

the

Data type: String

version number to check

version

Data type: String

permit_unsafe_versions

Data type: Boolean

peadm::bolt_version

Type: Ruby 4.x API

The peadm::bolt_version function.

peadm::bolt_version()

The peadm::bolt_version function.

Returns: Any

peadm::certname

Type: Puppet Language

This function accepts a variety of data types which could represent single targets, and returns the certname corresponding to the input.

For Target objects, or arrays of a single Target object, a "certname" var can be set, which determines that target's certname. Otherwise, the target's name is its certname. For strings, the certname is equal to the string. Undef input returns undef.

`peadm::certname(Variant[Target,

      String,
      Undef,
      Array[Target,1,1],
      Array[String,1,1],
      Array[Undef,1,1],
      Array[Any,0,0]] $target)`

This function accepts a variety of data types which could represent single targets, and returns the certname corresponding to the input.

For Target objects, or arrays of a single Target object, a "certname" var can be set, which determines that target's certname. Otherwise, the target's name is its certname. For strings, the certname is equal to the string. Undef input returns undef.

Returns: Variant[String, Undef]

target

Data type: Variant[Target, String, Undef, Array[Target,1,1], Array[String,1,1], Array[Undef,1,1], Array[Any,0,0]]

peadm::convert_hash

Type: Puppet Language

converts two arrays into hash

Examples

peadm::convert_hash(['type', 'status'], [['xl', 'running'], ['large', 'failed']])
[
  { type => xl, status => running}, { type => large, status => failed }
]

peadm::convert_hash(Array $keys, Array[Array] $values)

The peadm::convert_hash function.

Returns: Array

Examples

peadm::convert_hash(['type', 'status'], [['xl', 'running'], ['large', 'failed']])
[
  { type => xl, status => running}, { type => large, status => failed }
]

keys

Data type: Array

an array of key names to be merged into the hash

values

Data type: Array[Array]

data to be merged into an array with the keys

peadm::convert_status

Type: Puppet Language

Transforms a value in a human readable status with or without colors

Examples

With colors

peadm::convert_status(true) = "\e[32moperational\e[0m"

Without colors

peadm::convert_status(true, 0, false) = "operational"

Using integers where 1 of 2 services has failed

peadm::convert_status(1, 2, false) = "degraded"

Using integers where 2 of 2 services has failed

peadm::convert_status(2, 2, false) = "failed"

Using integers where 0 of 2 services has failed

peadm::convert_status(0, 2, false) = "operational"

peadm::convert_status(Variant[String,Boolean, Integer] $status, Optional[Integer] $total = 0, Optional[Boolean] $use_colors = true)

The peadm::convert_status function.

Returns: String A status as a string with or without color

Examples

With colors

peadm::convert_status(true) = "\e[32moperational\e[0m"

Without colors

peadm::convert_status(true, 0, false) = "operational"

Using integers where 1 of 2 services has failed

peadm::convert_status(1, 2, false) = "degraded"

Using integers where 2 of 2 services has failed

peadm::convert_status(2, 2, false) = "failed"

Using integers where 0 of 2 services has failed

peadm::convert_status(0, 2, false) = "operational"

status

Data type: Variant[String,Boolean, Integer]

A value of true, false, degraded, or an Integer that represents number of non operationally services If using an integer, you must also supply the total amount of services

total

Data type: Optional[Integer]

the total number of services, used only when the status is an integer

use_colors

Data type: Optional[Boolean]

Adds colors to the status, defaults to true

peadm::determine_status

Type: Puppet Language

}

Examples

peadm::determine_status($data, true)
{
 "failed" => {
           "activity/pe-std-replica.puppet.vm" => false,
         "classifier/pe-std-replica.puppet.vm" => false,
   "file-sync-client/pe-std-replica.puppet.vm" => false,
             "master/pe-std-replica.puppet.vm" => false,
           "puppetdb/pe-std-replica.puppet.vm" => false,
               "rbac/pe-std-replica.puppet.vm" => false
 },
 "passed" => {
            "activity-service/pe-std.puppet.vm" => true,
              "broker-service/pe-std.puppet.vm" => true,
          "classifier-service/pe-std.puppet.vm" => true,
        "code-manager-service/pe-std.puppet.vm" => true,
    "file-sync-client-service/pe-std.puppet.vm" => true,
   "file-sync-storage-service/pe-std.puppet.vm" => true,
        "orchestrator-service/pe-std.puppet.vm" => true,
                   "pe-master/pe-std.puppet.vm" => true,
             "puppetdb-status/pe-std.puppet.vm" => true,
                "rbac-service/pe-std.puppet.vm" => true
 },
  "state" => {
            "activity-service/pe-std.puppet.vm" => true,
            "activity/pe-std-replica.puppet.vm" => false,
              "broker-service/pe-std.puppet.vm" => true,
          "classifier-service/pe-std.puppet.vm" => true,
          "classifier/pe-std-replica.puppet.vm" => false,
        "code-manager-service/pe-std.puppet.vm" => true,
    "file-sync-client-service/pe-std.puppet.vm" => true,
    "file-sync-client/pe-std-replica.puppet.vm" => false,
   "file-sync-storage-service/pe-std.puppet.vm" => true,
              "master/pe-std-replica.puppet.vm" => false,
        "orchestrator-service/pe-std.puppet.vm" => true,
                   "pe-master/pe-std.puppet.vm" => true,
             "puppetdb-status/pe-std.puppet.vm" => true,
            "puppetdb/pe-std-replica.puppet.vm" => false,
                "rbac-service/pe-std.puppet.vm" => true,
                "rbac/pe-std-replica.puppet.vm" => false
 },
 "status" => "\e[33mdegraded\e[0m"

peadm::determine_status(Array $status_data, Boolean $use_colors = true)

}

Returns: Hash A simplified hash of of status data for the given stack

Examples

peadm::determine_status($data, true)
{
 "failed" => {
           "activity/pe-std-replica.puppet.vm" => false,
         "classifier/pe-std-replica.puppet.vm" => false,
   "file-sync-client/pe-std-replica.puppet.vm" => false,
             "master/pe-std-replica.puppet.vm" => false,
           "puppetdb/pe-std-replica.puppet.vm" => false,
               "rbac/pe-std-replica.puppet.vm" => false
 },
 "passed" => {
            "activity-service/pe-std.puppet.vm" => true,
              "broker-service/pe-std.puppet.vm" => true,
          "classifier-service/pe-std.puppet.vm" => true,
        "code-manager-service/pe-std.puppet.vm" => true,
    "file-sync-client-service/pe-std.puppet.vm" => true,
   "file-sync-storage-service/pe-std.puppet.vm" => true,
        "orchestrator-service/pe-std.puppet.vm" => true,
                   "pe-master/pe-std.puppet.vm" => true,
             "puppetdb-status/pe-std.puppet.vm" => true,
                "rbac-service/pe-std.puppet.vm" => true
 },
  "state" => {
            "activity-service/pe-std.puppet.vm" => true,
            "activity/pe-std-replica.puppet.vm" => false,
              "broker-service/pe-std.puppet.vm" => true,
          "classifier-service/pe-std.puppet.vm" => true,
          "classifier/pe-std-replica.puppet.vm" => false,
        "code-manager-service/pe-std.puppet.vm" => true,
    "file-sync-client-service/pe-std.puppet.vm" => true,
    "file-sync-client/pe-std-replica.puppet.vm" => false,
   "file-sync-storage-service/pe-std.puppet.vm" => true,
              "master/pe-std-replica.puppet.vm" => false,
        "orchestrator-service/pe-std.puppet.vm" => true,
                   "pe-master/pe-std.puppet.vm" => true,
             "puppetdb-status/pe-std.puppet.vm" => true,
            "puppetdb/pe-std-replica.puppet.vm" => false,
                "rbac-service/pe-std.puppet.vm" => true,
                "rbac/pe-std-replica.puppet.vm" => false
 },
 "status" => "\e[33mdegraded\e[0m"

status_data

Data type: Array

Raw json data as returned by puppet infra status --format=json

use_colors

Data type: Boolean

Adds colors to the status, defaults to true

peadm::fail_on_transport

Type: Puppet Language

Fails if any nodes have the chosen transport.

Useful for excluding PCP when it's not appopriate

peadm::fail_on_transport(TargetSpec $nodes, String $transport)

Fails if any nodes have the chosen transport.

Useful for excluding PCP when it's not appopriate

Returns: Any

nodes

Data type: TargetSpec

transport

Data type: String

peadm::file_content_upload

Type: Ruby 4.x API

The peadm::file_content_upload function.

peadm::file_content_upload(String[1] $content, String[1] $destination, TargetOrTargets *$targets)

The peadm::file_content_upload function.

Returns: Any

content

Data type: String[1]

destination

Data type: String[1]

*targets

Data type: TargetOrTargets

peadm::file_or_content

Type: Puppet Language

The peadm::file_or_content function.

peadm::file_or_content(String $param_name, Variant[String, Undef] $file, Variant[String, Undef] $content)

The peadm::file_or_content function.

Returns: Any

param_name

Data type: String

file

Data type: Variant[String, Undef]

content

Data type: Variant[String, Undef]

peadm::flatten_compact

Type: Puppet Language

The peadm::flatten_compact function.

peadm::flatten_compact(Array $input)

The peadm::flatten_compact function.

Returns: Any

input

Data type: Array

peadm::generate_pe_conf

Type: Puppet Language

Generate a pe.conf file in JSON format

peadm::generate_pe_conf(Hash $settings)

The peadm::generate_pe_conf function.

Returns: String

settings

Data type: Hash

A hash of settings to set in the config file. Any keys that are set to undef will not be included in the config file.

peadm::get_targets

Type: Puppet Language

Accept undef or a SingleTargetSpec, and return an Array[Target, 1, 0]. This differs from get_target() in that:

• It returns an Array[Target, 1, 0], rather than a Target
• It will accept undef and return [ ].

peadm::get_targets(Variant[TargetSpec, Undef] $spec, Optional[Integer[1,1]] $count = undef)

Accept undef or a SingleTargetSpec, and return an Array[Target, 1, 0]. This differs from get_target() in that:

• It returns an Array[Target, 1, 0], rather than a Target
• It will accept undef and return [ ].

Returns: Any

spec

Data type: Variant[TargetSpec, Undef]

count

Data type: Optional[Integer[1,1]]

peadm::node_manager_yaml_location

Type: Ruby 4.x API

The peadm::node_manager_yaml_location function.

peadm::node_manager_yaml_location()

The peadm::node_manager_yaml_location function.

Returns: Any

peadm::oid

Type: Puppet Language

The peadm::oid function.

peadm::oid(String $short_name)

The peadm::oid function.

Returns: Any

short_name

Data type: String

peadm::plan_step

Type: Ruby 4.x API

The peadm::plan_step function.

peadm::plan_step(String $step_name, Callable &$block)

The peadm::plan_step function.

Returns: Any

step_name

Data type: String

&block

Data type: Callable

peadm::recovery_opts_default

Type: Puppet Language

The peadm::recovery_opts_default function.

peadm::recovery_opts_default()

The peadm::recovery_opts_default function.

Returns: Any

peadm::wait_until_service_ready

Type: Puppet Language

A convenience function to help remember port numbers for services and handle running the wait_until_service_ready task

peadm::wait_until_service_ready(String $service, TargetSpec $target)

A convenience function to help remember port numbers for services and handle running the wait_until_service_ready task

Returns: Any

service

Data type: String

target

Data type: TargetSpec

Data types

Peadm::Ldap_config

The Peadm::Ldap_config data type.

Alias of

Struct[{
  base_dn                             => String,
  connect_timeout                     => Integer,
  disable_ldap_matching_rule_in_chain => Boolean,
  display_name                        => String,
  group_lookup_attr                   => String,
  group_member_attr                   => String,
  group_name_attr                     => String,
  group_object_class                  => String,
  Optional[group_rdn]                 => Optional[String],
  Optional[help_link]                 => Optional[String],
  hostname                            => String,
  Optional[login]                     => Optional[String],
  Optional[password]                  => Optional[String],
  port                                => Integer,
  search_nested_groups                => Boolean,
  ssl                                 => Boolean,
  ssl_hostname_validation             => Boolean,
  ssl_wildcard_validation             => Boolean,
  start_tls                           => Boolean,
  user_display_name_attr              => String,
  user_email_attr                     => String,
  user_lookup_attr                    => String,
  Optional[user_rdn]                  => Optional[String],
}]

Peadm::Pe_version

The Peadm::Pe_version data type.

Alias of

Pattern[/^\d+\.\d+\.\d+(-.+)?$/]

Peadm::Pem

The Peadm::Pem data type.

Alias of

Pattern[/^-----BEGIN/]

Peadm::Recovery_opts

The Peadm::Recovery_opts data type.

Alias of

Struct[{
  'orchestrator' => Optional[Boolean],
  'puppetdb'     => Optional[Boolean],
  'rbac'         => Optional[Boolean],
  'activity'     => Optional[Boolean],
  'ca'           => Optional[Boolean[false]],
  'classifier'   => Optional[Boolean],
}]

Peadm::SingleTargetSpec

A SingleTargetSpec represents any String, Target or single-element array of one or the other that can be passed to gettargets() to return an Array[Target, 1, 1]. This is a constrained type variant of Boltlib::TargetSpec for use when a _single target is valid, but multiple targets are not.

Alias of

Variant[Pattern[/\A[^[:space:],]+\z/], Target, Array[Peadm::SingleTargetSpec, 1, 1]]

Tasks

agent_install

Install the Puppet agent from a master

Supports noop? false

Parameters

server

Data type: String

The resolvable name of the Puppet server to install from

install_flags

Data type: Array[String]

Positional arguments to pass to the shell installer

agent_upgrade

Upgrade the target system using upgrade.bash from a master

Supports noop? false

Parameters

server

Data type: String

The resolvable name of the Puppet server to upgrade from

backup_classification

A task to call the classification api and write to file

Supports noop? false

Parameters

directory

Data type: String

The directory to write the classification output to. Directory must exist

cert_data

Return certificate data related to the Puppet agent

Supports noop? false

cert_valid_status

Check primary for valid state of a certificate

Supports noop? false

Parameters

certname

Data type: String

The certifcate name to check validation of

code_manager

Perform various code manager actions

Supports noop? false

Parameters

action

Data type: String

What code manager action to perform. For example: 'deploy production'; 'flush-environment-cache'; 'file-sync commit'

code_sync_status

A task to confirm code is in sync accross the cluster for clusters with code manager configured

Supports noop? false

Parameters

environments

Data type: Array

A list of environments to check, pass a single value of all for all

divert_code_manager

Divert the code manager live-dir setting

Supports noop? false

download

Download a file using curl

Supports noop? false

Parameters

source

Data type: String

Where to download the file from

path

Data type: String

Where to save the downloaded file

verify_download

Data type: Boolean

Whether to check the integrity of the downloaded file

key_server

Data type: String

The GPG keyserver to retrieve GPG keys from

enable_replica

Execute the enable replica puppet command

Supports noop? false

Parameters

replica

Data type: String

The name of the replica to enable

token_file

Data type: Optional[String]

The name of the token file to use for auth

filesize

Return the size of a file in bytes

Supports noop? false

Parameters

path

Data type: String

Path to the file to return the size of

get_peadm_config

Run on a PE primary node to return the currently configured PEAdm parameters

Supports noop? false

infrastatus

Runs puppet infra status and returns the output

Supports noop? false

Parameters

format

Data type: Enum[json,text]

The type of output to return

mkdir_p_file

Create a file with the specified content at the specified location

Supports noop? false

Parameters

path

Data type: String

The fully qualified path of the file to create

content

Data type: String

The content to create the file with

owner

Data type: Optional[String]

The file owner

group

Data type: Optional[String]

The file group

mode

Data type: Optional[String]

The file mode

chown_r

Data type: Optional[String]

If supplied, recursively chown starting at this path

mv

Wrapper task for mv command

Supports noop? false

Parameters

source

Data type: String

Current path of file

target

Data type: String

New path of file

pe_install

Install Puppet Enterprise from a tarball

Supports noop? false

Parameters

tarball

Data type: String

The path to the Puppet Enterprise tarball

peconf

Data type: Optional[String]

The path to the pe.conf file

install_extra_large

Data type: Optional[Boolean]

If true, optimize task for known manual issues with extra-large installs. Do not use for upgrades

puppet_service_ensure

Data type: Optional[Enum['stopped']]

If 'stopped', ensure the Puppet agent is not running when install completes

pe_ldap_config

Set the ldap config in the PE console

Supports noop? false

Parameters

ldap_config

Data type: Peadm::Ldap_config

The hash of options for ldap.

pe_main

Data type: String

The PE Main server

pe_uninstall

Uninstall Puppet Enterprise

Supports noop? false

precheck

Return pre-check information about a system

Supports noop? false

provision_replica

Execute the replica provision puppet command

Supports noop? false

Parameters

replica

Data type: String

The name of the replica to provision

token_file

Data type: Optional[String]

The name of the token-file for auth

legacy

Data type: Boolean

Set to true if provisioning a replica for PE 2019.5 or older. Defaults to false

puppet_infra_upgrade

Execute the puppet infra upgrade command

Supports noop? false

Parameters

type

Data type: Enum[compiler,replica]

Which kind of infra node to upgrade

targets

Data type: Array[String]

The certnames of the targets to upgrade

token_file

Data type: Optional[String]

The path to the token file to use

wait_until_connected_timeout

Data type: Integer

How many seconds to wait for targets to be connected to the orchestrator

puppet_runonce

Run the Puppet agent one time

Supports noop? false

Parameters

noop

Data type: Optional[Boolean]

If true, run Puppet in no-op mode

rbac_token

Get and save an rbac token for the root user, admin rbac user

Supports noop? false

Parameters

password

Data type: String

The password for the admin user

read_file

Read the contents of a file

Supports noop? false

Parameters

path

Data type: String

Path to the file to read

restore_classification

A short description of this task

Supports noop? false

Parameters

classification_file

Data type: String

The full path to a backed up or transformed classification file

sign_csr

Submit a certificate signing request

Supports noop? false

Parameters

certnames

Data type: Array[String]

A list of certnames to sign

ssl_clean

Clean an agent's certificate

Supports noop? false

Parameters

certname

Data type: String

The certname to clean

submit_csr

Submit a certificate signing request

Supports noop? false

Parameters

dns_alt_names

Data type: Optional[Array[String]]

DNS Alternative Names to request for the certificate

transform_classification_groups

Transform the user groups from a source backup to a list of groups on the target server

Supports noop? false

Parameters

source_directory

Data type: String

Location of Source node group yaml file

working_directory

Data type: String

Location of target node group yaml file and where to create the transformed file

wait_until_service_ready

Return when the orchestrator service is healthy, or timeout after 15 seconds

Supports noop? false

Parameters

service

Data type: Enum[all, ca, pe-master, orchestrator-service]

What service to check. For example: all, pe-master, orchestrator-service

port

Data type: Enum['8140', '8143']

Which port to query the status API on

Plans

peadm::add_database

The peadm::add_database class.

Parameters

The following parameters are available in the peadm::add_database plan:

• targets
• primary_host
• mode
• begin_at_step

targets

Data type: Peadm::SingleTargetSpec

primary_host

Data type: Peadm::SingleTargetSpec

mode

Data type: Optional[Enum['init', 'pair']]

Default value: undef

begin_at_step

Data type: Optional[Enum[ 'init-db-node', 'replicate-db', 'update-classification', 'update-db-settings', 'cleanup-db', 'finalize']]

Default value: undef

peadm::convert

This plan sets required certificate extensions on PE nodes, and configures the required PE node groups to make an existing cluster compatible with management using PEAdm.

Parameters

The following parameters are available in the peadm::convert plan:

• primary_host
• replica_host
• compiler_hosts
• primary_postgresql_host
• replica_postgresql_host
• compiler_pool_address
• internal_compiler_a_pool_address
• internal_compiler_b_pool_address
• dns_alt_names
• begin_at_step

primary_host

Data type: Peadm::SingleTargetSpec

replica_host

Data type: Optional[Peadm::SingleTargetSpec]

Default value: undef

compiler_hosts

Data type: Optional[TargetSpec]

Default value: undef

primary_postgresql_host

Data type: Optional[Peadm::SingleTargetSpec]

Default value: undef

replica_postgresql_host

Data type: Optional[Peadm::SingleTargetSpec]

Default value: undef

compiler_pool_address

Data type: String

Default value: $primary_host

internal_compiler_a_pool_address

Data type: Optional[String]

Default value: undef

internal_compiler_b_pool_address

Data type: Optional[String]

Default value: undef

dns_alt_names

Data type: Array[String]

Default value: [ ]

begin_at_step

Data type: Optional[Enum[ 'modify-primary-certs', 'modify-infra-certs', 'convert-node-groups', 'finalize']]

Default value: undef

peadm::install

Install a new PE cluster

Parameters

The following parameters are available in the peadm::install plan:

• compiler_pool_address
• internal_compiler_a_pool_address
• internal_compiler_b_pool_address
• pe_installer_source
• ldap_config
• primary_host
• replica_host
• compiler_hosts
• primary_postgresql_host
• replica_postgresql_host
• console_password
• version
• dns_alt_names
• pe_conf_data
• r10k_remote
• r10k_private_key_file
• r10k_private_key_content
• deploy_environment
• license_key_file
• license_key_content
• stagingdir
• download_mode
• permit_unsafe_versions

compiler_pool_address

Data type: Optional[String]

The service address used by agents to connect to compilers, or the Puppet service. Typically this is a load balancer.

Default value: undef

internal_compiler_a_pool_address

Data type: Optional[String]

A load balancer address directing traffic to any of the "A" pool compilers. This is used for DR configuration in large and extra large architectures.

Default value: undef

internal_compiler_b_pool_address

Data type: Optional[String]

A load balancer address directing traffic to any of the "B" pool compilers. This is used for DR configuration in large and extra large architectures.

Default value: undef

pe_installer_source

Data type: Optional[String]

The URL to download the Puppet Enterprise installer media from. If not specified, PEAdm will attempt to download PE installation media from its standard public source. When specified, PEAdm will download directly from the URL given.

Default value: undef

ldap_config

Data type: Optional[Peadm::Ldap_config]

If specified, configures PE RBAC DS with the supplied configuration hash. The parameter should be set to a valid set of connection settings as documented for the PE RBAC /ds endpoint. See: https://puppet.com/docs/pe/latest/rbac_api_v1_directory.html#put_ds-request_format

Default value: undef

primary_host

Data type: Peadm::SingleTargetSpec

replica_host

Data type: Optional[Peadm::SingleTargetSpec]

Default value: undef

compiler_hosts

Data type: Optional[TargetSpec]

Default value: undef

primary_postgresql_host

Data type: Optional[Peadm::SingleTargetSpec]

Default value: undef

replica_postgresql_host

Data type: Optional[Peadm::SingleTargetSpec]

Default value: undef

console_password

Data type: String

version

Data type: Peadm::Pe_version

Default value: '2019.8.8'

dns_alt_names

Data type: Optional[Array[String]]

Default value: undef

pe_conf_data

Data type: Optional[Hash]

Default value: { }

r10k_remote

Data type: Optional[String]

Default value: undef

r10k_private_key_file

Data type: Optional[String]

Default value: undef

r10k_private_key_content

Data type: Optional[Peadm::Pem]

Default value: undef

deploy_environment

Data type: Optional[String]

Default value: undef

license_key_file

Data type: Optional[String]

Default value: undef

license_key_content

Data type: Optional[String]

Default value: undef

stagingdir

Data type: Optional[String]

Default value: undef

download_mode

Data type: Enum[direct,bolthost]

Default value: 'bolthost'

permit_unsafe_versions

Data type: Boolean

Default value: false

peadm::modify_certificate

Certificates can be modified by adding extensions, removing extensions, or setting DNS alternative names.

Parameters

The following parameters are available in the peadm::modify_certificate plan:

• targets
• primary_host
• add_extensions
• remove_extensions
• dns_alt_names
• force_regenerate

targets

Data type: TargetSpec

primary_host

Data type: Peadm::SingleTargetSpec

add_extensions

Data type: Hash

Default value: { }

remove_extensions

Data type: Array

Default value: [ ]

dns_alt_names

Data type: Optional[Array]

Default value: undef

force_regenerate

Data type: Boolean

Default value: false

peadm::status

Return status information from one or more PE clusters in a table format

Examples

peadm::status($targets, 'table', true, true)

Parameters

The following parameters are available in the peadm::status plan:

• targets
• format
• summarize
• verbose
• colors

targets

Data type: TargetSpec

These are a list of the primary puppetservers from one or multiple puppet stacks

format

Data type: Enum[json,table]

The output format to dump to stdout (json or table)

Default value: 'table'

summarize

Data type: Boolean

Controls the type of json output to render, defaults to true

Default value: true

verbose

Data type: Boolean

Toggles the output to show all the operationally services, can be loads more data

Default value: false

colors

Data type: Boolean

Toggles the usage of colors, you may want to disable if the format is json

Default value: $format

peadm::upgrade

Upgrade a PEAdm-managed cluster

Parameters

The following parameters are available in the peadm::upgrade plan:

• compiler_pool_address
• internal_compiler_a_pool_address
• internal_compiler_b_pool_address
• pe_installer_source
• primary_host
• replica_host
• compiler_hosts
• primary_postgresql_host
• replica_postgresql_host
• version
• token_file
• stagingdir
• download_mode
• permit_unsafe_versions
• begin_at_step

compiler_pool_address

Data type: Optional[String]

The service address used by agents to connect to compilers, or the Puppet service. Typically this is a load balancer.

Default value: undef

internal_compiler_a_pool_address

Data type: Optional[String]

A load balancer address directing traffic to any of the "A" pool compilers. This is used for DR configuration in large and extra large architectures.

Default value: undef

internal_compiler_b_pool_address

Data type: Optional[String]

A load balancer address directing traffic to any of the "B" pool compilers. This is used for DR configuration in large and extra large architectures.

Default value: undef

pe_installer_source

Data type: Optional[String]

The URL to download the Puppet Enterprise installer media from. If not specified, PEAdm will attempt to download PE installation media from its standard public source. When specified, PEAdm will download directly from the URL given.

Default value: undef

primary_host

Data type: Peadm::SingleTargetSpec

replica_host

Data type: Optional[Peadm::SingleTargetSpec]

Default value: undef

compiler_hosts

Data type: Optional[TargetSpec]

Default value: undef

primary_postgresql_host

Data type: Optional[Peadm::SingleTargetSpec]

Default value: undef

replica_postgresql_host

Data type: Optional[Peadm::SingleTargetSpec]

Default value: undef

version

Data type: Peadm::Pe_version

token_file

Data type: Optional[String]

Default value: undef

stagingdir

Data type: String

Default value: '/tmp'

download_mode

Data type: Enum[direct,bolthost]

Default value: 'bolthost'

permit_unsafe_versions

Data type: Boolean

Default value: false

begin_at_step

Data type: Optional[Enum[ 'upgrade-primary', 'upgrade-node-groups', 'upgrade-primary-compilers', 'upgrade-replica', 'upgrade-replica-compilers', 'finalize']]

Default value: undef

Change log

All notable changes to this project will be documented in this file. The format is based on Keep a Changelog and this project adheres to Semantic Versioning.

v3.8.0 (2022-06-21)

Full Changelog

Added

• Fix classification when adding some components #258 (ody)
• Add task and update configure plan to allow for ldap configuration on… #253 (bwilcox)

Fixed

• Restart compiler services #271 (ody)
• Set additional rules on replica when adding compiler #270 (ody)
• Updates documentation #269 (ody)

v3.7.0 (2022-05-11)

Full Changelog

Added

• Add optional pe_installer_source parameter #261 (reidmv)
• Support PE 2021.6 and 2019.8.11 #260 (reidmv)
• Add experimental restore plan #250 (reidmv)

v3.6.0 (2022-05-04)

Full Changelog

Added

• Add experimental backup plan #235 (davidsandilands)

Fixed

• Add read map so puppetdb can start on new compiler #252 (ody)
• Fix and document recovery procedure for PSQL #243 (ody)

v3.5.0 (2022-04-13)

Full Changelog

Added

• Add support for SLES 12 #245 (MartyEwings)
• Add peadm::add_database plan #241 (ody)
• Add experimental peadm::util::code_sync_status plan #221 (reidmv)

Fixed

• Fix json escaping in bash_task_helper #246 (reidmv)
• Fix load balancer bug in peadm::get_peadm_config task #244 (reidmv)

v3.4.0 (2022-03-08)

Full Changelog

Added

• Allow compilers and replicas to merge csr_attributes. #238 (bwilcox)
• Add support for PE 2021.5 #237 (reidmv)
• Improve handling of unsuitable LANG settings #236 (reidmv)
• Add peadm::backup plan #226 (davidsandilands)

Fixed

• Make versions match SemVer pattern #232 (reidmv)

v3.3.0 (2022-01-05)

Full Changelog

Added

• Support PE 2021.4 #229 (reidmv)
• Add development and testing option to permit installing unsupported PE versions #204 (jarretlavallee)

Fixed

• Fail agent_install if agent is already installed #223 (reidmv)
• Catch mv errors when downloading #220 (reidmv)
• Determine validation key from asc signature file #219 (reidmv)
• Improve reliability of downloading PE tarball #215 (mcka1n)

v3.2.0 (2021-09-20)

Full Changelog

Added

• Add auto-generated REFERENCE.md documentation #211 (reidmv)
• Make PEAdm a Puppet supported module #199 (ody)

Fixed

• Update documentation to reference supported PE version #213 (reidmv)
• Fix output of peadm::status when used with multiple clusters #209 (reidmv)

v3.1.0 (2021-09-10)

Full Changelog

Added

• Support PE 2021.3 #203 (reidmv)
• Add PE download signature checking #201 (timidri)
• Add task to report on code synchronization status #196 (davidsandilands)
• Add an experimental peadm::uninstall plan #195 (mcka1n)
• Remove hardcoded default memory configuration #194 (reidmv)
• Highlight user-facing plans by hiding internal plans from bolt plan show output #189 (reidmv)
• Add get_peadm_config task #187 (reidmv)
• Replace plan peadm::modify_cert_extensions with peadm::modify_certificate #181 (reidmv)

Fixed

• Fix upgrade without replica #198 (reidmv)
• Fix upgrade bug for token files with newlines #193 (reidmv)
• Move load_balancer class to examples #183 (reidmv)
• Fix GitHub README.md problem #182 (reidmv)

v3.0.1 (2021-06-30)

Full Changelog

Fixed

• Add missing parenthesis to add_compiler plan #177 (timidri)
• Use absolute links so they render properly on the Forge #175 (binford2k)

v3.0.0 (2021-06-29)

Full Changelog

Changed

• Global rename of primary/replica and postgresql parameters #161 (timidri)
• Language and terminology updates #153 (davidsandilands)

Added

• Update workflow PE defaults to latest LTS #170 (reidmv)
• Add add_replica plan #166 (timidri)
• Support latest PE release #157 (ody)
• Add add_compiler plan #154 (timidri)

Fixed

• Resolving linting issues #165 (davidsandilands)
• Fix installer exit handling #152 (reidmv)

2.5.0

Summary

Changes

• Require WhatsARanjit-node_manager >= 0.7.5
• Require puppetlabs-stdlib >= 6.5.0

Improvements

• Support PE 2021.0
• Handle exit code 11 from replica upgrade task gracefully. Code 11 means "PuppetDB sync in progress but not yet complete"
• Further remediate the bug fixed in 2.4.2, by ensuring that all peadm-managed node groups preserve existing data or class parameters not explicitly being managed
• Switch dependency enumeration from in-project Puppetfile to bolt-project.yaml modules setting

2.4.5

Summary

Bugfix release

Bugfixes

• Fix an issue in the convert plan incorrectly disallowing conversion of deployments newer than 2019.7.0.
• Fix a problem with the Peadm::SingleTargetSpec type alias.
• Fix peadm::puppet_runonce to correctly return a failure if the Puppet agent run had resource failures.

2.4.4

Summary

Support PE 2019.8.4 and newer 2019.8.z releases

Improvements

• Validation should Permit installing or upgrading to any PE 2019.8.z release

2.4.3

Summary

Support PE 2019.8.3

Improvements

• Support installing or upgrading to PE 2019.8.3

2.4.2

Summary

Bugfix release

Bugfixes

• Previously, on upgrade, peadm could overwrite user configuration data on the PE Master group because it overwrote the entire configuration data value. This release modifies the peadm::setup::node_manager desired state configuration to merge required configuration into any existing configuration when configuring data on the PE Master node group.

2.4.1

Summary

Bugfix release

Bugfixes

• Previously, on upgrade, peadm did not ensure that PostgreSQL servers' pe.conf file contained the critical keys that inform the installer that the system is a stand-alone database. The peadm::upgrade plan now ensures the critical keys are correct as part of the upgrade preparation.
• When upgrading a DR replica to PE 2019.8.0 or 2019.8.1, there is an installer bug that causes the upgrade to fail due to how puppetdb delete-reports performs in this configuration. This release works around the problem by bypassing puppetdb delete-reports. This workaround will be removed in future releases of peadm after the installer / puppetdb delete-reports bug is fixed.

2.4.0

Summary

Readme updates and further convert plan efficiency improvements

Features

• In the peadm::convert plan, certificates which already contain requested extensions will not be re-issued. This will accelerate the convert process, or allow re-runs of the convert process to move more quickly.

Improvements

• The README now provides more detailed information on how customers using the peadm module should go about getting support for it.

2.3.0

Summary

Add ability to resume peadm::upgrade or peadm::convert at an intermediate step, rather than requiring re-runs to perform all plan actions from the beginning.

Features

• Added begin_at_step parameter and documentation to peadm::upgrade and peadm::convert

Bugfixes

• In peadm::convert plan, stop the Puppet agent before writing the csr_attributes.yaml file, to prevent possible agent interference
• In the peadm::convert plan during finalization, run the Puppet agent on the primary server first, then the rest, to avoid the possibility of a puppetserver restart impacting Puppet agent runs on other systems.

Improvements

• In the peadm::convert plan, when no peadm_availability_group trusted fact is present to identify if compilers should be members of the A pool or B pool, check for pp_cluster being used to designate this configuration before falling back to a simple even/odd split. This is to catch systems provisioned with the old pe_xl module, which used pp_cluster to designate A/B.

2.2.1

Summary

Bugfix release

Bugfixes

• Fixed problem with internal_compiler_b_pool_address parameter name in peadm::action::configure plan

2.2.0

Summary

Reliability fixes for 2019.8.1, README updates, and simpification of the convert plan. New parameters added for internal_compiler_a_pool_address and internal_compiler_b_pool_address to configure lb addresses for each half of the compiler pool, so that this configuration does not need to be re-applied after upgrades.

Features

• Added parameters to configure compiler pool addresses for the A and B availability groups. These are used in large and extra large architectures.
• Add basic informational messages to upgrade plan output, to communicate when different stages of the upgrade begin.

Bugfixes

• Fixed GH-118, wherein a compiler would unnecessarily send duplicate work to an extra configured PuppetDB endpoint.
• Puppet infra upgrade operations now always wait until target nodes are connected before attempting an operation

Improvements

• Provide a useful overview of the module in the README so that readers can quickly gain a sense of how the module is used, what it affects, and what it does not affect.
• Eliminate configure_node_groups parameter to peadm::convert. Perform the correct action(s) automatically.

Release 2.1.1

Summary

Development tool and README fixes.

Bugfixes

• Remove reference to Puppet Support team from README. This module is intended to be used in collaboration with Professional Services and Solutions Architects at Puppet, not Support
• Fixes and improvements to Docker development tools

Release 2.1.0

Summary

Support upgrades from PE 2018.1 to 2019.7.

Features

• Support added for upgrading from PE 2018.1 to 2019.7

Release 2.0.0

Summary

Major version release to support PE 2019.7.

Users can use peadm 2.0.0 to create new 2019.7 deployments, or to upgrade from 2019.5 to 2019.7.

To deploy PE 2019.5 or older, use a 1.x release of peadm.

Features

• Support added for PE 2019.7

Release 1.2.0

Summary

Feature and bugfix release.

Features

• Add direct download option for PE installers (download_mode parameter)
• Add docker features for testing deployments in containers
• Improve idempotency around CSR submission and signing
• Add basic version validation

Bugfixes

• Make peadm::read_file compatible with python3 for better CentOS 8 support
• Fix failure to install when passing passing r10k_private_key parameters
• Improve error handling of peadm::download task

Release 1.1.0

Summary

This release supports PE 2019.1 through 2019.5.

A Changelog was not maintained prior to this release.

Features

• Provision new PE clusters with standard, large, or extra-large architecture
• Upgrade PE clusters provisioned with peadm

Bugfixes

N/A

This changelog is used track changes with this module in human readable format. Feel free to reference tickets with links or other important information the reader would find useful when determining the level of risk with upgrading. For more information on changelogs please see the keeping a changelog site.

* This Changelog was automatically generated by github_changelog_generator