pe_quick_data
Version information
This version is compatible with:
- Puppet Enterprise 2023.2.x, 2023.1.x, 2023.0.x, 2021.7.x, 2021.6.x, 2021.5.x, 2021.4.x, 2021.3.x, 2021.2.x, 2021.1.x, 2021.0.x, 2019.8.x, 2019.7.x, 2019.5.x, 2019.4.x, 2019.3.x, 2019.2.x, 2019.1.x, 2019.0.x, 2018.1.x, 2017.3.x, 2016.4.x
- Puppet >= 4.10.12 < 8.0.0
- , , ,
Plans:
- data_collect
Start using this module
Add this module to your Puppetfile:
mod 'puppetlabs-pe_quick_data', '2.3.4'
Learn more about managing modules with a PuppetfileDocumentation
Module for quick data collection of Puppet Enterprise environment
Description
The plan in this repo will quickly collect data from a Puppet Enterprise server to be used for understanding the environment better and providing the customer with value as a baseline moving forward. Plans and tasks included in this module can be run using either Puppet Bolt or Puppet Enterprise Plans and/or Tasks. Thanks to the repository at https://github.com/puppetlabs/pe_tech_check for providing support script parts of the data collect.
For a review of the data collected as part of the module please see the documentation located at https://github.com/puppetlabs/puppetlabs-pe_quick_data/blob/master/data_use/data_use.md
WARNING: When running this module and have the puppet metrics collector module running, ensure that the puppet metrics module is at version 6.1.0 or later. If not, file sizes of the pe_quick_data output will typically be >10 GBs in size or larger. To avoid this issue, upgrade the puppetlabs/puppet_metrics_collector to version 6.1.0 or higher when using version 6 of the collector. Versions 5.3.0 and below are not affected.
Requirements
- If running the module from an admin workstation, Puppet Bolt version 2 or higher when using Bolt to execute the plan via command line
- See https://puppet.com/docs/bolt/latest/bolt_installing.html for installation and Bolt usage
- Puppet Enterprise master server with bash capabilities for collecting data
- The module can be run from the Puppet Enterprise 2019 or higher console through PE Plans
- Root/su authentication to the PE Master that data is collected from is required
Setup for using the module - Bolt
-
Use module from the forge by adding the module to your Boltdir directory Puppetfile.
mod 'puppetlabs-pe_quick_data', '2.3.4'
-
Run
bolt puppetfile install
from the Boltdir directory where the Puppetfile is located. -
Save inventory.yaml file with PE master defined for the environment into the same Boltdir directory.
-
The default directory for the data collection is currently /var/tmp/pe_quick_data on the Puppet Enterprise master, but this can be changed at run time.
-
If an alternate directory is specified for the data collection through the outputdir parameter, the folder pe_quick_data will be created on the PE master, in the alternate directory.
Setup for using the module - Puppet Enterprise (PE) 2019 or higher
- Use the module from the forge by adding the module to your Puppetfile in the appropriate environment.
- Deploy the updated Puppetfile to the environment via your current deployment method.
- Run the pe_quick_data::data_collect plan using Plans in PE.
- Specify the PE master as the target and optionally specify the output_dir location.
Bolt Plan Use
To use the plan run bolt plan run pe_quick_data::data_collect
with --targets specified to point to the master or masters for Puppet Enterprise
Parameters for use with the plan
output_dir - specifies the directory where the pe_quick_data directory will be created.
- This uses tar and gzip to zip the data into a file and will be left in the output_dir.
- The default directory is /var/tmp/ and a tar.gz file is placed within pe_quick_data directory in the default directory.
- It is an optional parameter at run time. If not specified, the default directory output will be used.
enable_logs - specifies whether or not to include the support script log output
- To enable, specifiy enable_logs=true when running the plan
- It is an optional parameter at run time. If not specified, support script logs are not included.
- If enabled_logs=true, the size of the output will be increased to a potentially large size.
download - allows for downloading to the Boltdir/downloads directory from where the plan is run
- To download the file to the Boltdir/downloads directory, specifiy download=true when running the plan
- It is an optional parameter at run time. If not specified, the file will not be downloaded to Boltdir/download
- Be aware that downloading the file to the Boltdir/downloads directory will delete all other downloads in that directory
- The download parameter requires version 2.20 of bolt or higher
Plan Use Examples
Run data collection with default output directory
bolt plan run pe_quick_data::data_collect --targets puppetm.example.com
Run data collection with default output directory with no inventory.yaml
bolt plan run pe_quick_data::data_collect --targets puppetm.example.com --user <USER> --private-key <KEY_PATH> --transport ssh --no-host-key-check --run-as root
Run data collection specifying an alternate output_dir
bolt plan run pe_quick_data::data_collect --targets puppetm.example.com output_dir=/tmp/
Run data collection and include support script logging
bolt plan run pe_quick_data::data_collect --targets puppetm.example.com enable_logs=true
What are plans?
Modules can contain plans that take action outside of a desired state managed by Puppet. It’s perfect for troubleshooting or deploying one-off changes, distributing scripts to run across your infrastructure, or automating changes that need to happen in a particular order as part of an application deployment.
Changelog
All notable changes to this project will be documented in this file.
Release v2.3.4 (2021-10-01)
Known Issues
Resolved Issues
Features
Compatibility with PE 2021
Release v2.3.1 (2020-09-29)
Known Issues
Puppet Metrics Collector increases output of data collection files to > 10GB in size
Resolved Issues
Corrected documentation versioning
Features
Release v2.3.0 (2020-09-29)
Known Issues
Puppet Metrics Collector increases output of data collection files to > 10GB in size
Resolved Issues
Features
Added collection of agent versions and count on the Puppet Enterprise server by OS
Release v2.2.0 (2020-09-03)
Resolved Issues
Features
Added the ability to download the gzip file from the Puppet Infrastructure server environment to the bolt client
Release v2.1.0 (2020-08-17)
Resolved Issues
Features
Moved all module collection to the site module task and removed task roles_and_profiles
Moved to discovering modulepaths instead of assuming the path for better results
Made tasks hidden from task show and documentation to alleviate some confusion as to what to run for data collection
Started to use pe_quick_data.log to log information for troubleshooting and quick information on the site_modules task
Bugfixes
Known Issues
Release v2.0.2 (2020-07-24)
Resolved Issues
Fixed documentation issues related to the Readme
Features
Bugfixes
Known Issues
Release v2.0.0 (2020-07-24)
Resolved Issues
Features
Moved puppet db queries to curl requests to support SSL, move away from requiring access tokens, and to support non-monolithic environments
Added additional query to gather resource usage by environment
Added support for gathering data for PE 2016 environments
Bugfixes
Known Issues
Release v1.2.3 (2020-7-17)
Resolved Issues
Corrected the module documentation for correct module based on version number
Release v1.2.1 (2020-7-15)
Resolved Issues
Deleting all folders out of the pe_quick_data folder when running the collect task. This is done to avoid an issue when removing other directories later in the task
Fixed issue where tar.gz files would compound in size adding any zip files in the output directory to a previous zip file
Release v1.2.0 (2020-7-10)
Resolved Issues
Some instances of Puppet Enterprise code environments may not always be located in the default location, thus the environments would not be found. Corrected to locate directories via a search for the codedir.
Features
Added an argument called enable_logs that defaults to not capturing the output of the support logs.
Bug Fixes
Release v1.1.0 (2020-06-26)
Resolved Issues
Corrected documentation with correct links for bolt docs
Features
Added site-module data task to the module
Bug Fixes
Corrected node count data collect to run appropriate queries based on PE version
Release v1.0.2 (2020-06-17)
Resolved Issues
Fixed documentation to indicate correct setup of module
Added the data collection markdown and data_use links and content to github
Features
Bugfixes
Known Issues
Release v1.0.1 (2020-06-16)
This is the inital release of the module which adds puppet queries to the already included 'puppet enteprise support' command included with Puppet Enterprise.
Features
Adds data in the support script collection that includes;
Number of roles and profiles Number of total resources utilized within the PE install Number of total resources per node Total Active Nodes Node Generic Operating System (OS) information
Bugfixes
Known Issues
Release [v1.0.1]
This is the inital release of the module which adds puppet queries to the already included 'puppet enteprise support' command included with Puppet Enterprise.
Features
Adds data in the support script collection that includes;
Number of roles and profiles Number of total resources utilized within the PE install Number of total resources per node Total Active Nodes Node Generic Operating System (OS) information
Bugfixes
Known Issues