local_rpm_repo

Table of Contents

1. Description
2. Setup - The basics of getting started with local_rpm_repo
   • What local_rpm_repo affects
   • Setup requirements
3. Usage - Configuration options and additional functionality
4. Limitations - OS compatibility, etc.
5. Development - Guide for contributing to the module

Description

This module allows you to create and deploy a http based RPM repository using a tar of RPMs as a source or an external puppet fileserver. This can be helpful for environments where some number of locally created and maintained RPMs must be distributed to a set of systems.

We use this to automatically maintain a simple repo containing a ppc64 build of puppet-agent.

Setup

What local_rpm_repo affects

It creates an apache vhost on the target machine and schedules a cron job of createrepo for each repository specified in the config. The repos will be within the specified vhost in their own folders.

Setup Requirements - Tar Source

When an http or https URL is provided the module assumes you have a simple web server configured to host a tar file for each repository you intend to deploy.

Tar files for each repository should be created and named with a format similar to ${reponame}-${version}.tar. For example, a repo named test-repo with a version string of 1.6.3.0 would have a corresponding tar named test-repo-1.6.3.0.tar. It is possible to override the ${reponame} portion of this string with an alternate prefix. See the reference documentation for detail.

Setup Requirements - Puppet Source

Configure a custom file server mount point to host the source data for your repositories. The base_url should contain a directory corresponding to the repository name or the prefix override configured for that repository. Within that directory there should be a folder for the target version which contains the desired RPMs in the expected deployment structure.

Usage

Tar Sources

The example below will configure three repos named repo1, repo2, and repo3 respectively on the target node. This node will expect connections to arrive to the vhost yum.example.com.

When puppet is applying this configuration it will expect to be able to download the following three files:

• http://bin.example.com/data/repo1-5.0.3.1.tar
• http://bin.example.com/data/repo2-1.0.7.0.tar
• http://bin.example.com/data/repo3-dev-1.0.0.0.tar

class local_rpm_repo {
  base_url_default => 'http://bin.example.com/data/',
  mirror_fqdn      => 'yum.example.com',
  repo_list        => {
    'repo1' => {
      'source_type' => 'tar',
      'version' => '5.0.3.1',
    },
    'repo2' => {
      'source_type' => 'tar',
      'version' => '1.0.7.0',
    },
    'repo3' => {
      'source_type' => 'tar',
      'version' => '5.0.3.1',
      'prefix'  => 'repo3-dev',
    },
  },
}

Alternatively the same configuration can be achieved via hiera:

---
local_rpm_repo::base_url_default: 'http://bin.example.com/data/'
local_rpm_repo::mirror_fqdn: 'yum.example.com'
local_rpm_repo::repo_list:
  repo1:
    source_type: 'tar'
    version: '5.0.3.1'
  repo2:
    source_type: 'tar'
    version: '1.0.7.0'
  repo3:
    source_type: 'tar'
    prefix: 'repo3-dev'
    version: '1.0.0.0'

Once applied, the following changes will be implemented:

The /var/yumrepos directory will contain the following sub folders:

• /var/yumrepos/repo1/versions/5.0.3.1/
• /var/yumrepos/repo2/versions/1.0.7.0/
• /var/yumrepos/repo3/versions/1.0.0.0/

The contents of the three tar files detailed above will be extracted to these directories.

An Apache vhost will be created listening at yum.example.com serving the /var/yumrepos directory.

Puppet Fileserver

This method uses the source directive on the puppet file resource type to recursively replicate the contents of the specified source folder as the repo version.

---
local_rpm_repo::base_url_default: 'http://bin.example.com/data/'
local_rpm_repo::mirror_fqdn: 'yum.example.com'
local_rpm_repo::repo_list:
  repo1:
    source_type: 'tar'
    version: '5.0.3.1'
  repo2:
    source_type: 'puppet'
    base_url: 'puppet:///external_files'
    version: '1.0.7.0'

In the example above repo1 will be declared from the tar http://bin.example.com/data/repo1-5.0.3.1.tar as normal, however, repo2 will replicate the contents of puppet:///external_files/repo2/1.0.7.0 to it's version directory.

Reposync

This method schedules a daily execution of reposync via cron to pull down the latest package files from an external repository. A separate job will be created for each CPU architecture and stored within the version folder used by other sync methods.

---
local_rpm_repo::base_url_default: 'http://bin.example.com/data/'
local_rpm_repo::mirror_fqdn: 'yum.example.com'
local_rpm_repo::repo_list:
  repo1:
    source_type: 'tar'
    version: '5.0.3.1'
  ibm-xl-compiler-runtime-7:
    source_type: 'reposync'
    base_url: 'ftp://public.dhe.ibm.com/software/server/POWER/Linux/rte/xlcpp/be/rhel7'
    gpgcheck: false,
    array_arch:
      - 'ppc64'
      - 'ppc64le'

Note that gpg configuration is supported in the context of this repository using the original key files from the remote repository.

Currently only publicly reachable repositories can be configured.

Wget

This method uses wget to recursively download the *.rpm files from a particular URL. It is intended to serve as a backup for repositories which have bad meta-data but good packages (Looking at you IBM).

---
local_rpm_repo::base_url_default: 'http://bin.example.com/data/'
local_rpm_repo::mirror_fqdn: 'yum.example.com'
local_rpm_repo::repo_list:
  repo1:
    source_type: 'tar'
    version: '5.0.3.1'
  ibm-power-tools:
    source_type: 'wget'
    base_url: 'https://public.dhe.ibm.com/software/server/POWER/Linux/yum/OSS/RHEL/7/'
    cut_dir_count: 7

Limitations

• Currently GPG signing of custom repositories is not supported.

Development

TBD.

Contributors

Check out the contributor list.

Reference

Table of Contents

Classes

• local_rpm_repo: Creates and hosts the specified RPM repositories.

Defined types

• local_rpm_repo::puppet_source: Repo with rpm files sourced from a puppet file server.
• local_rpm_repo::reposync_source: Repo with rpm files sourced via a reposync call.
• local_rpm_repo::tar_source: Repo with instance data retrieve from a tar file.
• local_rpm_repo::wget_source: Repo with rpm files sourced via a reposync call.

Data types

• Local_rpm_repo::PuppetUrl: Matches puppet file server source url strings
• Local_rpm_repo::RepoData: Repo configuration file used by reposync to download the data
• Local_rpm_repo::RepoGPGKey: GPG Key file definition
• Local_rpm_repo::RepoUrl: Valid source URL types for reposync
• Local_rpm_repo::SourceUrl: URL types for tar source data.
• Local_rpm_repo::Version: Allowed version semantics on our data.

Classes

local_rpm_repo

A simple class to create and manage locally maintained repositories using tar file containing the RPMs.

Examples

Definition of three repositories

class { 'local_rpm_repo':
  base_url_default => 'http://bin.example.com/data/',
  mirror_fqdn      => 'yum.example.com',
  repo_list        => {
    'repo1' => { version => '1.0.0.0' },
    'repo2' => { version => '1.0.0.0' },
    'repo3' => { version => '1.0.0.0' },
  },
}

Parameters

The following parameters are available in the local_rpm_repo class:

• mirror_fqdn
• base_url_default
• repo_base_path
• repo_base_cache_path
• repo_list
• suppress_cron_stdout

mirror_fqdn

Data type: Stdlib::Fqdn

Fully Qualified Domain Name which will be used as the apache vhost name on this server. Note that this FQDN should resolve to this server.

base_url_default

Data type: Optional[Local_rpm_repo::SourceUrl]

This specifies the URL from which the repo tar should be extracted or the remote files should be replicated if a specific url is not provided in repo_list. When used, the application assumes the source data is located within the specified directory.

For example, given a repo named 'test-repo' at version '1.0.5.1':

• http://bin.example.org/data - expects http://bin.example.org/data/test-repo-1.0.5.1.tar to exist
• puppet:///repo_files - expects the folder structure 'puppet:///repo_files/test-repo/1.0.5.1'

repo_base_path

Data type: Stdlib::AbsolutePath

Absolute path to the parent directory of the yum repository

Default value: '/var/yumrepos'

repo_base_cache_path

Data type: Stdlib::AbsolutePath

Absolute path to the parent directory of the yum repository cache directory.

Default value: '/var/cache/yumrepos'

repo_list

Data type: Hash[String, Local_rpm_repo::RepoData]

A hash where the keys are the names of the repos to be deployed. The following options are supported for each repo:

• version - version of the source repo file
• source_prefix - prefix of the file, defaults to the repo name.

@note At least one repository must be specified

Default value: {}

suppress_cron_stdout

Data type: Boolean

When enabled stdout generated by the createrepo command(s) will be directed to /dev/null. This is very helpful if these messages would be emailed unnecessarily in your environment.

Default value: true

Defined types

local_rpm_repo::puppet_source

This type is an implementation detail and should not be declared directly!

Parameters

The following parameters are available in the local_rpm_repo::puppet_source defined type:

• version
• base_url
• prefix
• versions_path

version

Data type: Local_rpm_repo::Version

Version number to instantiate.

base_url

Data type: Local_rpm_repo::PuppetUrl

Puppet path to the RPM directory. See the repo::base_url_default for further details.

prefix

Data type: String

String prepended to the version number to determine the source tar file name. Note that this field is a namevar.

Default value: $name

versions_path

Data type: Stdlib::AbsolutePath

Absolute path to the versions subdirectory for this repository.

local_rpm_repo::reposync_source

This type is an implementation detail and should not be declared directly!

Parameters

The following parameters are available in the local_rpm_repo::reposync_source defined type:

• base_url
• versions_path
• gpgcheck
• gpgkey
• array_arch
• description
• initial_timeout

base_url

Data type: Local_rpm_repo::RepoUrl

Source against which the reposync command will execute

versions_path

Data type: Stdlib::AbsolutePath

Absolute path to the versions subdirectory for this repository. This will be used as the target for the reposync command.

gpgcheck

Data type: Boolean

When true, this module will attempt to initialize the gpgkey file & will ensure it is valid during this configuration

gpgkey

Data type: Optional[Local_rpm_repo::RepoGPGKey]

This parameter must be included when gpgcheck is enabled. Includes the partial name of the keyfile as well as it's desired content.

Default value: undef

array_arch

Data type: Array[String]

List of processor architectures to iterate through.

Default value: []

description

Data type: String

When present, the config file can have a different "name" subkey in its INI.

Default value: $name

initial_timeout

Data type: Integer

Execution timeout on the first synchronization of the repository.

Default value: 300

local_rpm_repo::tar_source

This type is an implementation detail and should not be declared directly!

Parameters

The following parameters are available in the local_rpm_repo::tar_source defined type:

• version
• base_url
• prefix
• versions_path

version

Data type: Local_rpm_repo::Version

Version number to instantiate.

base_url

Data type: Variant[Stdlib::Httpurl, Stdlib::Httpsurl]

Location containing the source tar file. See the repo::base_url_default for further details.

prefix

Data type: String

String prepended to the version number to determine the source tar file name. Note that this field is a namevar.

Default value: $name

versions_path

Data type: Stdlib::AbsolutePath

Absolute path to the versions subdirectory for this repository.

local_rpm_repo::wget_source

This type is an implementation detail and should not be declared directly!

Parameters

The following parameters are available in the local_rpm_repo::wget_source defined type:

• base_url
• versions_path
• cut_dir_count
• initial_timeout

base_url

Data type: Local_rpm_repo::RepoUrl

Source against which the reposync command will execute

versions_path

Data type: Stdlib::AbsolutePath

Absolute path to the versions subdirectory for this repository. This will be used as the target for the wget command.

cut_dir_count

Data type: Integer

When wget recursively downloads the contents of a webpage the default behavior is to create a folder for each subdirectory along the path. This parameter controls the number of directories to cut making the resulting tree more concise.

initial_timeout

Data type: Integer

Execution timeout on the first synchronization of the repository.

Default value: 300

Data types

Local_rpm_repo::PuppetUrl

Matches puppet file server source url strings

Alias of

Pattern[/(?i:^puppet?:\/\/)/]

Local_rpm_repo::RepoData

Repo configuration file used by reposync to download the data

Alias of

Variant[Struct[
    source_type        => Enum['puppet'],
    version            => Local_rpm_repo::Version,
    Optional[prefix]   => String,
    Optional[base_url] => Local_rpm_repo::PuppetURL,
  ], Struct[
    source_type        => Enum['tar'],
    version            => Local_rpm_repo::Version,
    Optional[prefix]   => String,
    Optional[base_url] => Local_rpm_repo::SourceUrl,
  ], Struct[
    source_type               => Enum['reposync'],
    base_url                  => Local_rpm_repo::RepoUrl,
    gpgcheck                  => Boolean,
    array_arch                => Array[String],
    Optional[gpgkey]          => Local_rpm_repo::RepoGPGKey,
    Optional[description]     => String,
    Optional[initial_timeout] => Integer,
  ], Struct[
    source_type               => Enum['wget'],
    base_url                  => Local_rpm_repo::RepoUrl,
    cut_dir_count             => Integer,
    Optional[initial_timeout] => Integer,
  ]]

Parameters

The following parameters are available in the Local_rpm_repo::RepoData data type:

• source_type
• version
• baseurl
• enabled
• gpgcheck
• array_arch
• description
• initial_timeout

source_type

Determines the type of source from which the repo content will be retrieved.

version

Puppet and tar type repositories use these version numbers in their deployment model.

baseurl

Source from which the repo content will be downloaded.

enabled

See man yum.conf

gpgcheck

See man yum.conf

array_arch

Reposync based repositories can be multi-architecture

description

Human readable string used in the declaration of the repository.

initial_timeout

Controls the command timeout via exec when the sync runs the first time. Not all modes support this parameter. Note that it defaults to 300 seconds when left out.

Local_rpm_repo::RepoGPGKey

GPG Key file definition

Alias of

Struct[=>]

Parameters

The following parameters are available in the Local_rpm_repo::RepoGPGKey data type:

• name
• content

name

Filename of the resulting keyfile

content

Raw content of the keyfile

Local_rpm_repo::RepoUrl

Valid source URL types for reposync

Alias of

Variant[Stdlib::Httpurl, Stdlib::Httpsurl, Pattern[/(?i:^ftp?:\/\/)/]]

Local_rpm_repo::SourceUrl

URL types for tar source data.

Alias of

Variant[Stdlib::Httpurl, Stdlib::Httpsurl]

Local_rpm_repo::Version

Allowed version semantics on our data.

Alias of

Pattern[/\d[.]\d[.]\d[.]\d/]

Changelog

All notable changes to this project will be documented in this file.

1.1.0 (2022-07-29)

Feature (1 change)

• feat: check and remove files which fail gpg (merge request)

1.0.0 (2022-07-29)

Feature (4 changes)

• feat: configurable first sync timeouts (merge request)
• feat: only sync the newest version (merge request)
• feat: wget based repository sources (merge request)
• feat: reposync based repository mirror source (merge request)

Maintenance (3 changes)

• maint: add changelog generation instructions (merge request)
• maint: convert to struct (merge request)
• maint: pdk update (merge request)

Release 0.2.1

Full Changelog

Features

• Allow cron output to be suppressed
• Updated to PDK 1.13.0

Release 0.2.0

Full Changelog

Features

• Using types for parameters.
• PDK 1.9.1
• Support for puppet custom mount points (file servers)

Bugfixes

• Switched to a refresh-only topology - nolonger has idempotent executions of commands

Release 0.1.2

Full Changelog

Bugfixes

• Added checks to prevent the source files from being downloaded even if the repo is already deployed. Note that it doesn't validate the contents of the repo so if it gets corrupted you'll need to delete.

Release 0.1.1

Full Changelog

Bugfixes

• Used a series of commands instead of tidy to cleanup old versions as the initial release would simply alternate between removing everything and creating everything.

Release 0.1.0

Features

• Initial release!