Puppet Rclone management module

Puppet module helping with installation and configuration of Rclone

Disclaimer: This module is in early stage of development - everything are subject to change. Any feedback or help is welcomed.

Description

Use this module to automate installation and configuration of Rclone - cloud rsync tool.

Setup

Setup Requirements

Depends on following modules:

• puppet/archive

Beginning with rclone

class { 'rclone':
  ensure => 'desired rclone version'
}

# configure s3
rclone::config::s3 { 'my_S3':
  access_key_id     => 'AKI...',
  secret_access_key => '...',
  os_user           => 'my_user',
  region            => 'eu-west-1',
}

Usage

See reference guide

Limitations

Module in early stage, anything can break anytime.

Development

Prerequisites: Installed ruby, gem and bundler and also PDK can help.

pdk bundle update
pdk bundle install

Run unit tests

pdk bundle exec rake spec

or with PDK

pdk test unit

Generate reference

bundle exec rake strings:generate:reference

Releasing

Build and release the module to puppet forge (note currently validation and changelog doesn't work)

pdk build
pdk release

Tag the release in git

git tag -a v<version> -m "message"
git push --tags

Bump the version to next devel and make pull request.

Reference

Table of Contents

Classes

Public Classes

• rclone: Download and install Rclone

Private Classes

• rclone::install: Ensures Rclone installed
• rclone::uninstall: Removes rclone installed by this module

Defined types

• rclone::config: General configuration for Rclone.
• rclone::config::gdrive: Google Drive configuration for Rclone.
• rclone::config::s3: S3 configuration for Rclone.

Functions

Public Functions

Private Functions

• rclone::last_version: Fetches the last released Rclone version from internet

Classes

rclone

Install rclone binary and man page

Examples

include rclone

Parameters

The following parameters are available in the rclone class:

• ensure

ensure

Data type: Pattern[/absent/, /latest/, /\d+\.\d+\.\d+/]

installed version, can be 'latest', 'absent' or valid version string

Default value: 'latest'

Defined types

rclone::config

Ensures Rclone configuration of given name, type and params. Include of rclone is required.

Examples

rclone::config { 'my_remote':
  os_user => 'my_user',
  type    => 'ftp',
  options => {...}
}

Parameters

The following parameters are available in the rclone::config defined type:

• ensure
• os_user
• type
• options
• config_name

ensure

Data type: Enum['present', 'absent']

configuration ensure

Default value: 'present'

os_user

Data type: String

operating system user - used to execute rclone commands, effective configuration owner

type

Data type: Enum[ 'amazon cloud drive', 'azureblob', 'b2', 'box', 'crypt', 'cache', 'chunker', 'drive', 'dropbox', 'fichier', 'ftp', 'google cloud storage', 'google photos', 'http', 'swift', 'hubic', 'jottacloud', 'koofr', 'local', 'mailru', 'mega', 'memory', 'onedrive', 'opendrive', 'pcloud', 'premiumizeme', 'putio', 'qingstor', 's3', 'sftp', 'sharefile', 'sugarsync', 'union', 'webdav', 'yandex']

configuration remote type

options

Data type: Hash[String, Optional[String]]

configuration options - Hash of options for rclone config command

config_name

Data type: String

configuration name - should be unique among Rclone configurations, defaults to title

Default value: $title

rclone::config::gdrive

Ensures Drive Rclone configuration of given name and params. Include of rclone is required. Support only service account credentials (token authentication requires human interaction when setup).

Examples

rclone::config::gdrive { 'drive_remote':
  os_user                     => 'my_user',
  client_id                   => 'SOME_CLIENT_ID',
  client_secret               => 'SOME_CLIENT_SECRET',
  service_account_credentials => 'SERVICE_CREDENTIALS',
}

Parameters

The following parameters are available in the rclone::config::gdrive defined type:

• ensure
• os_user
• config_name
• client_id
• client_secret
• service_account_credentials
• scope
• root_folder_id
• team_drive

ensure

Data type: Enum['present', 'absent']

configuration ensure

Default value: 'present'

os_user

Data type: String

operating system user - used to execute rclone commands, effective configuration owner

config_name

Data type: String

configuration name - should be unique among Rclone configurations, defaults to title

Default value: $title

client_id

Data type: String

Google drive client_id, maps to Rclone client_id property

client_secret

Data type: String

Google drive client_secret, maps to Rclone client_secret property

service_account_credentials

Data type: String

Google drive service_account_credentials, maps to Rclone service_account_credentials property

scope

Data type: String

Google drive access scope, maps to Rclone scope property

Default value: 'drive'

root_folder_id

Data type: Optional[String]

Id of the drive root folder, maps to Rclone root_folder_id property

Default value: undef

team_drive

Data type: Optional[String]

Id of the team drive, maps to Rclone team_drive property

Default value: undef

rclone::config::s3

Ensures S3 Rclone configuration of given name and params. Include of rclone is required. Currently only AWS provider is supported.

Examples

rclone::config::s3 { 's3_remote':
  os_user           => 'my_user',
  access_key_id     => 'SOME_ACCESS_KEY',
  secret_access_key => 'SECRET_ACCESS_KEY',
  region            => 'us-east-1',
}

Parameters

The following parameters are available in the rclone::config::s3 defined type:

• ensure
• os_user
• config_name
• access_key_id
• secret_access_key
• region
• s3_provider
• canned_acl
• endpoint
• location_constraint
• location_constraint
• server_side_encryption
• storage_class

ensure

Data type: Enum['present', 'absent']

configuration ensure

Default value: 'present'

os_user

Data type: String

operating system user - used to execute rclone commands, effective configuration owner

config_name

Data type: String

configuration name - should be unique among Rclone configurations, defaults to title

Default value: $title

access_key_id

Data type: String

S3 provider's access_key_id, maps to Rclone access_key_id property

secret_access_key

Data type: String

S3 provider's secret_access_key, maps to Rclone secret_access_key property

region

Data type: String

S3 provider's region, maps to Rclone region property

s3_provider

Data type: Enum['AWS']

S3 provider, maps to Rclone provider property

Default value: 'AWS'

canned_acl

Data type: Optional[String]

S3 canned ACL, maps to Rclone acl property

Default value: undef

endpoint

Data type: Optional[String]

S3 provider's endpoint, maps to Rclone endpoint property

Default value: undef

location_constraint

Data type: Optional[String]

S3 location_constraint, maps to Rclone location_constraint property

Default value: undef

location_constraint

S3 location_constraint, maps to Rclone location_constraint property

Default value: undef

server_side_encryption

Data type: Optional[String]

S3 server_side_encryption, maps to Rclone server_side_encryption property

Default value: undef

storage_class

Data type: Optional[String]

S3 storage_class, maps to Rclone storage_class property

Default value: undef

Functions

Changelog

All notable changes to this project will be documented in this file.

Release 0.1.0

Features

Bugfixes

Known Issues