duplicity

#duplicity

##Overview

Configure duply on top of duplicity to provide a profile-based, easy to use backup and restore system.

##Usage

Install duplicity and duply with all default values.

class { 'duplicity':
  ensure => present,
}

Install a more recent version of duply from the sourceforge project page

class { 'duplicity':
  duply_package_provider => 'archive',
  duply_archive_version  => '1.9.1',
  duply_archive_md5sum   => 'd584940b9c740c81a2a081bc154084b9',
}

Specify the backup server to be used; see the duplicity documentation for more information about the available protocols.

class { 'duplicity':
  backup_target_url      => 'ftps://backup.example.com/',
  backup_target_username => 'username',
  backup_target_password => 'password',
}

In case you're using duply 1.10+ and a storage backend that requires additional environment variables to be set, use the following pattern

class { 'duplicity':
  duply_environment => [
    "export AWS_ACCESS_KEY_ID='${my_access_key}'",
    "export AWS_SECRET_ACCESS_KEY='${my_secret_key}'",
  ],
}

This works on a profile-level as well.

Configure a simple backup profile that stops an application before the backup starts and starts it when complete. It will run once a day, do incremental backups by default and create a full backup if the previous full backup is older than 7 days. Duplicity will keep at most two full backups and purge older ones.

duplicity::profile { 'system':
  full_if_older_than  => '7D',
  max_full_backups    => 2,
  cron_hour           => '4',
  cron_minute         => '0',
  exec_before_content => '/bin/systemctl stop myapp',
  exec_after_content  => '/bin/systemctl start myapp',
}

Backup a file and restore it from a previous backup if it is not existing. Setting ensure to backup will only backup the file but not restore it.

Note: a directory will only be restored if the directory is not existing - an empty directory is not replaced.

duplicity::file { '/path/to/file':
  ensure => present,
}

Backup a directory by using a specific backup profile and exclude a bunch of files.

$data_dir = '/var/lib/jira'

duplicity::file { $data_dir:
  profile => 'jira',
  exclude => [
    "${data_dir}/caches",
    "${data_dir}/tmp",
    "${data_dir}/plugins/.osgi-plugins/felix/felix-cache",
    "${data_dir}/plugins/.osgi-plugins/transformed-plugins",
  ],
}

Define a GnuPG key pair BEEF1234 to be used to de/encrypt the backup on the node itself and configure the backup profile to use it. The encrytion key ALICE00001 is used to decrypt the backup on another node (e.g. the admin's workstation).

duplicity::private_key { 'BEEF1234':
  content => hiera('duplicity::private_key::BEEF1234'),
}

duplicity::public_key { 'BEEF1234':
  content => template('path/to/BEEF1234.pub.asc.erb'),
}

class { 'duplicity':
  gpg_encryption_keys => ['ALICE00001', 'BEEF1234'],
  gpg_signing_key     => 'BEEF1234',
}

Or turn off the encryption of backups for a particular profile altogether:

duplicity::profile { 'system':
  gpg_encryption => false,
}

##Limitations

The module has been tested on the following operating systems. Testing and patches for other platforms are welcome.

• Debian 7.0 (Wheezy)
• Debian 8.0 (Jessie)
• Ubuntu 14.04 (Trusty Tahr)
• Ubuntu 16.04 (Xenial Xerus)
• RHEL/Centos 6

##Contributing

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request

###Development

This project uses rspec-puppet and beaker to ensure the module works as expected and to prevent regressions.

gem install bundler
bundle install --path vendor

bundle exec rake spec
bundle exec rake beaker

(note: see Beaker - Supported ENV variables for a list of environment variables to control the default behaviour of Beaker)

2017-01-04 - Release 4.3.0

The max_age parameter can be set in the profile (#33).

2016-12-06 - Release 4.2.0

Allow the Duply batch command to be customized, e.g. change the used operators or add additional commands (#31).

2016-11-27 - Release 4.1.0

Summary

This release ships two major changes

• Improve compatibility with duply 1.10+ which removed the environment variable handling required by many storage backends, e.g. NoAuthHandlerFound: No handler was ready to authenticate. (S3) (#25)
• Add support to control the niceness (#28)

Upgrade notes

Duply 1.10+ removed support for many backend-specific environment variables (see changelog):

... and instead opted for removing almost all code that deals with special env vars required by backends. adding and modifying these results in too much overhead so i dropped this feature. the future alternative for users is to consult the duplicity manpage and add the needed export definitions to the conf file.

In order to upgrade to this version, it is now up to the client to provide the environment variables. This can be done on a class or profile level. The specified variables are stored in duply's configuration file. Example:

class { 'duplicity':
  duply_environment => [
    "export AWS_ACCESS_KEY_ID='${my_access_key}'",
    "export AWS_SECRET_ACCESS_KEY='${my_secret_key}'",
  ],
}

Minor

A couple of minor updates have been applied

• Consistent operating system names for osfamily checks (#29)
• Add Ubuntu 16.04 to test matrix, remove Ubuntu 12.04
• Run beaker tests on travis CI

##2016-05-30 - Release 4.0.0 ###Summary

This release adds compatibility with Puppet 4 (#19, #20).

Note: As part of the upgrade the abandoned module rodjek/logrotate was replaced with yo61/logrotate.

##2016-04-25 - Release 3.6.3 ###Summary

Add missing resource dependency in duplicity::file (#24).

##2016-04-24 - Release 3.6.2 ###Summary

Fix No content, source or symlink specified warnings caused by concat when neither $exec_before_content nor $exec_before_source are specified (same for $exec_after_*, #23).

##2016-04-17 - Release 3.6.1 ###Summary

Update cron job to use correct purgeFull/purge-full command on CentOS 6 (#22).

##2016-03-13 - Release 3.6.0 ###Summary

This release fixes a bug in the duplicity::profile's cron job when using a version of duply that is lower than 1.7.1. The bug affects the default system package provided by Debian 7 / Ubuntu 14.04 and below.

Furthermore, the pre and post scripts run before and after a backup can be replaced with custom ones.

• Add the ability to specify before/after script source/content in a profile (#21)
• Fix pre 1.7.1 error (purgeFull vs. purge-full) (#9)

The unit and acceptance test infrastructure has been updated as well:

• Test only against the latest Puppet 3.x version
• Test against Puppet 4.x (experimental, Puppet 4 is currently not supported due to issues with logrotate)
• Replace VirtualBox with Docker

##2016-01-10 - Release 3.5.2 ###Summary

Add scope to profile_exec_* calls so that duplicity works with Puppet 4 (#18).

##2015-12-30 - Release 3.5.1 ###Summary

Fixes a bug which causes an error when trying to restore a duplicity::file' file resource whose path contains whitespaces. This issue effects the path parameter - the root - of the duplicity::file resource only, e.g.

duplicity::file { '/path/with/white space': }

Any path that is part of an inclusion list is NOT effected by the problem.

##2015-10-18 - Release 3.5.0 ###Summary

Allow configuration of the Duply cache (ARCH_DIR) directory (#17).

##2015-09-09 - Release 3.4.0 ###Summary

Add ability to pass extra options to duplicity (#12).

##2015-07-21 - Release 3.3.0 ###Summary

Add extra option that allows to download duply using a proxy server (#11).

##2015-07-11 - Release 3.2.0 ###Summary

The duply executable is no longer referenced directly but sourced from the PATH environment variable instead (#10). As a result of this change, the duply_executable parameter has been deprecated and was replaced with the more specific exec_path and duply_archive_executable parameters.

####Further changes

• Improved compatibility with Puppet 4 (logrotate #46).

##2015-04-15 - Release 3.1.1 ###Summary

Fix up too restrictive permission on files contained in the tarball that was published to the forge during the last release (see #8).

##2015-04-12 - Release 3.1.0 ###Summary

By default, all backups are encrypted via GPG. With the new boolean gpg_encryption parameter, this behaviour can be turned off in case encryption is not necessary. It can be configured on a per-profile level (see #3).

##2015-03-31 - Release 3.0.1 ###Summary

Just a bugfix release that fixes the wrong duply_executable being used.

##2015-03-31 - Release 3.0.0 ###Summary

This release adds a better separation of the configuration parameters when using the package vs. archive resource type.

Furthermore, it adds support for RHEL/CentOS 6.

####Changes to default behavior

• Package provider archive: the duply archive is now downloaded directly from sourceforge instead some dude's dropbox account; using HTTP by default to facilitate the usage of a HTTP proxy
• Package provider archive: the ambigious usage of the duply_package_ensure parameter to transport the default version and the packate state has been replaced with the duply_archive_version parameter
• On RedHat-based systems, the yum package is preferred over the sourceforge one

####Further changes

• Documentation of the duplicity class has been reviewed
• Using the archive no longer requires to specify duply_package_name or duply_archive_url; setting duply_archive_version and duply_archive_md5sum is enough
• Add support for Ubuntu 12.04 and 14.04
• Add system tests

##2014-12-25 - Release 2.0.0 ###Summary

• Add default package name/version/handler for Debian based distributions
• Replace gini/archive with camptocamp/archive
• Fix broken example in README