github_inventory
Version information
This version is compatible with:
- Puppet Enterprise 2023.2.x, 2023.1.x, 2023.0.x, 2021.7.x, 2021.6.x, 2021.5.x, 2021.4.x, 2021.3.x, 2021.2.x, 2021.1.x, 2021.0.x, 2019.8.x, 2019.7.x, 2019.5.x, 2019.4.x, 2019.3.x, 2019.2.x, 2019.1.x, 2019.0.x, 2018.1.x, 2017.3.x, 2017.2.x, 2016.4.x
- Puppet >= 4.10.0 < 8.0.0
- , , , , , , ,
Tasks:
- resolve_reference
Plans:
- clone_git_repos
- count
- latest_semver_tags
- required_checks
- workflows
Start using this module
Add this module to your Puppetfile:
mod 'bolterrific-github_inventory', '0.2.1'
Learn more about managing modules with a PuppetfileDocumentation
github_inventory
Table of Contents
Description
github_inventory is an inventory reference plugin for Puppet Bolt. It
uses the GitHub API to dynamically provide a collection of local
transport
Targets that represent each repository under a GitHub organization or user.
This module also contains an example Bolt project with a working
inventory.yaml
and several Bolt plans.
Why is this useful?
Modeling GitHub repositories as Inventory Targets lets admins orchestrate org-wide repo operations at scale, using simple & reusable Bolt Plans.
How is this useful with Bolt?
Simplicity, speed, and reusability: It's trivial to define an inventory, and
Bolt plans are generally concise and legible. And when using multiple Targets
(repos) with Bolt plan functions like run_*()
/parallelize()
, you get
concurrent execution built-in for free. Plans are easy to wrap up in a module
and share, so the next time someone has to reset all the required PR checks in
a 200-repo org, they can just pull out the plan and run it on targets from
their org.
Each github_inventory
Target describes its repository in its .facts
, using
the same keys as the data structure returned by GitHub's /orgs/{org}/repos
or /repos/{username}/repos
endpoints (with one important
exception).
Targets use the local
transport by default. This keeps execution as
frictionless as possible and ensures that a known version of Ruby (the Bolt
interpreter's) is available to execute tasks. It also also opens the door
to advanced uses, like git-cloning repos en masse into local working
directories and enforcing file conventions with Tasks and apply()
blocks.
Note: Targets don't use the remote
Transport. It can
only run remote Tasks (which rules out the built-in http_request
) and
can't compile Puppet apply()
blocks.
Setup
Setup Requirements
- Puppet Bolt 3.0+ or 2.37, installed from an OS package (don't use the RubyGem)
- A GitHub API personal auth token with sufficient scope
- The
octokit
RubyGem
Beginning with github_inventory
-
If you are using rvm, you must disable it before running bolt:
rvm use system
-
Install the RubyGem dependencies using Bolt's
gem
commandOn most platforms:
/opt/puppetlabs/bolt/bin/gem install --user-install -g gem.deps.rb
On Windows:
"C:/Program Files/Puppet Labs/Bolt/bin/gem.bat" install --user-install -g gem.deps.rb
-
(If using the example Bolt plans in this module)
Set the environment variables
GITHUB_ORG
andGITHUB_API_TOKEN
to appropriate values for your GitHub organization. The API token needs a scope that can query the organization's repos.
Usage
To use this plugin in your own Bolt project, configure it to provide targets
in the inventory file.
Using the plugin in a Bolt inventory file
An example inventory.yaml
file:
groups:
- name: repo_targets
targets:
- _plugin: github_inventory # <- Plugin provides `local` Targets
org: simp # <- GitHub org with Target repos
github_api_token: # <- API token with scope that can get repos
_plugin: env_var # <- (provided by another Bolt plugin)
var: GITHUB_API_TOKEN
config:
transport: local
local:
interpreters:
.rb: /opt/puppetlabs/bolt/bin/ruby
tmpdir:
_plugin: env_var
var: PWD
Reference
See REFERENCE.md
Limitations
Targets' .facts
keys use _name
instead of name
github_inventory
Target facts use the key_name
instead of the repo-levelname
key.- This avoids compilation errors in Puppet
apply()
blocks.
Limitations when running the example project
In order to provide the example bolt project in the same module as the
inventory plugin, modules/github_inventory/
is symlinked to the repo's
top-level directory. This allows the bolt project to find the inventory
plugin, but there are some side-effects:
-
Bolt will emit a (benign) warning near the beginning of each run:
The project 'github_inventory' shadows an existing module of the same name [ID: project_shadows_module]
-
Git may not establish the symlink properly for Windows users
This quirk only affects the example bolt project; it will not affect the
inventory plugin, or running the github_inventory::
plans from your own Bolt
project.
Development
Submit PRs on the project's GitHub page.
Reference
Table of Contents
Tasks
resolve_reference
: Return a GitHub organization's repositories as local inventory targets
Plans
github_inventory::clone_git_repos
: Clone all repos into a local directorygithub_inventory::count
: Example plan, prints number of Targets from inventorygithub_inventory::latest_semver_tags
: Report the highest SemVer tag for each repo (that has SemVer tags)github_inventory::required_checks
: List and/or set which PR checks are required on each repogithub_inventory::workflows
: Return repos with GitHub Actions workflows
Tasks
resolve_reference
Return a GitHub organization's repositories as local inventory targets
Supports noop? false
Parameters
org
Data type: String[1]
GitHub org name (or user login) with repos
github_api_token
Data type: Optional[String[1]]
Optional GitHub personal OAuth token, which may be useful to avoid the GitHub API's unauthenticated rate limits
archived_repos
Data type: Boolean
When true, includes archived repositories in results.
private_repos
Data type: Boolean
When true, includes private repositories in results.
allow_list
Data type: Optional[Array[String[1]]]
repo names/patterns to include in inventory, drops all other repos
block_list
Data type: Optional[Array[String[1]]]
repo names/patterns to reject from inventory (can reject targets in allow_list)
transport_type
Data type: String[1]
Bolt Transport type of repository Targets
extra_gem_path
Data type: Optional[String[1]]
Additional GEM_PATH path for ruby gems (to find octokit
)
Plans
github_inventory::clone_git_repos
Clone all repos into a local directory
Parameters
The following parameters are available in the github_inventory::clone_git_repos
plan:
targets
Data type: TargetSpec
Name of github_inventory
Targets (or inventory group)
Default value: 'github_repos'
target_dir
Data type: Stdlib::Absolutepath
Local directory to clone repos into
Default value: "${system::env('PWD')}/_repos"
github_inventory::count
Example plan, prints number of Targets from inventory
Parameters
The following parameters are available in the github_inventory::count
plan:
targets
Data type: TargetSpec
Name of github_inventory
Targets (or inventory group)
Default value: 'github_repos'
github_inventory::latest_semver_tags
Report the highest SemVer tag for each repo (that has SemVer tags)
- Note ONLY reports repos with SemVer tags (ignores
/^v/
and/-d$/
)
Parameters
The following parameters are available in the github_inventory::latest_semver_tags
plan:
targets
Data type: TargetSpec
Name of github_inventory
Targets (or inventory group)
Default value: 'github_repos'
github_api_token
Data type: Sensitive[String[1]]
GitHub API token. Doesn't require any scope for public repos.
Default value: (system::env('GITHUB_API_TOKEN'))
github_inventory::required_checks
List and/or set which PR checks are required on each repo
Parameters
The following parameters are available in the github_inventory::required_checks
plan:
targets
Data type: TargetSpec
Name of github_inventory
Targets (or inventory group)
Default value: 'github_repos'
github_api_token
Data type: Sensitive[String[1]]
GitHub API token. Needs repo
scope to see or set checks.
Default value: (system::env('GITHUB_API_TOKEN'))
checks
Data type: Optional[String[1]]
Optional comma-delimited list of required PR Checks to set on all repos If defined, this will overwrite ALL repos' required PR checks
Default value: undef
github_inventory::workflows
Return repos with GitHub Actions workflows
Parameters
The following parameters are available in the github_inventory::workflows
plan:
targets
Data type: TargetSpec
By default: github_repos
group from inventory
Default value: 'github_repos'
github_api_token
Data type: Sensitive[String[1]]
GitHub API token. Doesn't require any scope for public repos.
Default value: (system::env('GITHUB_API_TOKEN'))
What are tasks?
Modules can contain tasks that take action outside of a desired state managed by Puppet. It’s perfect for troubleshooting or deploying one-off changes, distributing scripts to run across your infrastructure, or automating changes that need to happen in a particular order as part of an application deployment.
Tasks in this module release
What are plans?
Modules can contain plans that take action outside of a desired state managed by Puppet. It’s perfect for troubleshooting or deploying one-off changes, distributing scripts to run across your infrastructure, or automating changes that need to happen in a particular order as part of an application deployment.
Changelog
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog.
0.2.1
Fixed
- Fixed bug that caused String
$targets
to fail in the example plangithub_inventory::workflows
(by addingget_targets()
)
Changed
- Renamed example
inventory.yaml
group togithub_repos
to play more nicely with other projects'inventory.yaml
0.2.0
Added
- New example plan,
github_inventory::git_clone
, which clones repo Targets into a local directory.
Changed
- Renamed Target fact
name
to_name
to prevent Boltapply()
Puppet compiles from failing withCannot reassign variable '$name'
errors
Fixed
- Fixed syntax and logic bugs in example plan
github_inventory::count
0.1.0
Added
- Initial project
- Inventory plugin that returns GitHub org repos as
local
transport Targets - Example Bolt project with working Plans and
inventory.yaml