Version information
This version is compatible with:
- Puppet Enterprise 2023.8.x, 2023.7.x, 2023.6.x, 2023.5.x, 2023.4.x, 2023.3.x, 2023.2.x, 2023.1.x, 2023.0.x, 2021.7.x, 2021.6.x, 2021.5.x, 2021.4.x, 2021.3.x, 2021.2.x, 2021.1.x, 2021.0.x, 2019.8.x, 2019.7.x, 2019.5.x, 2019.4.x, 2019.3.x, 2019.2.x, 2019.1.x, 2019.0.x, 2018.1.x, 2017.3.x, 2017.2.x, 2017.1.x, 2016.5.x, 2016.4.x
- Puppet >= 3.8.0
Start using this module
Add this module to your Puppetfile:
mod 'mikmanzato-yabt', '1.2.0'
Learn more about managing modules with a PuppetfileDocumentation
yabt
Table of Contents
Module Description
Yabt is the Yet-another backup tool, a simple but effective backup tool for GNU/Linux systems. This Puppet module simplifies the task of installing Yabt and creating configurations to manage backup jobs.
Setup
What yabt affects
This module affects just the yabt configuration files and directories.
Beginning with yabt
To have Puppet install Yabt with the default parameters, declare the yabt class:
class { '::yabt': }
You can also configure some global defaults related to notifications and dump or backup destinations:
class { '::yabt':
dump_location => 'ftps://user:pass@ftp.example.com/',
rdiffbackup_destination => 'user@example.com::',
notifications_enabled => true,
notifications_from => 'yabt@example.com',
notifications_recipients => 'me@example.com,you@example.com',
notifications_smtp_hostname => 'localhost',
sn_recurrence => 'daily',
}
All global configuration will end up in the main yabt configuration file, /usr/local/etc/yabt/main.conf.
Then you configure specific dump or backup jobs. Configuration for jobs will end up in the job configuration directory, /usr/local/etc/yabt/jobs.d/. Each job configuration file will be named after the job.
This is a rdiff-backup job of a local directory /home/user:
::yabt::rdiffbackup_job { 'yabt_home_user':
source => '/home/user',
}
which will use the global rdiffbackup_destination
configured above.
A MySQL dump job looks like this:
::yabt::mysqldb_dump_job { 'yabt_mysqldump_myapp':
user => 'mysql_user',
password => 'mysql_pass',
hostname => 'mysql_host',
dbname => 'mysql_dbname',
storage_dir => "/var/lib/myapp/storage",
}
which will use the global dump_location
configured above.
Usage
This section highlights the main classes and the basic configuration. Additional configuration options are documented in the reference section.
Initialization
Class ::yabt
defines common settings.
class { '::yabt':
dump_location => 'ftps://username:password@host/dir',
duplicity_target_url => 'ftp://username:password@host/dir',
rdiffbackup_destination => 'username@hostname::',
notifications_enabled => true,
notifications_from => 'yabt@example.com',
notifications_recipients => 'yourname@example.com',
sn_recurrence => 'daily',
}
Directory dump job
The ::yabt::dir_dump_job
resource dumps a local directory in "tar" format.
::yabt::dir_dump_job { 'job_name':
path => "path_to_directory",
}
Etc directory dump job
The ::yabt::etc_dir_dump_job
resource backs up the /etc directory.
include ::yabt::etc_dir_dump_job
It is a special case of the directory dump job.
Subversion repositories dump job
The ::yabt::svn_dump_job
resource backs up local subversion repositories.
::yabt::svnrepositories_dump_job { 'svnrepositories':
parent_path => '/var/svnroot',
}
All repositories in directory parent_path
will be backed up.
MySQL database dump job
The ::yabt::mysqldb_dump_job
resource dumps a MySQL database. The dump is
performed using mysqldump
which must be installed on the local system.
::yabt::mysqldb_dump_job { 'job_name':
user => 'mysql_username',
password => 'mysql_password',
hostname => 'mysql_server_host',
dbname => 'name_of_db',
}
If the database references data files which are stored on the filesystem one can backup the filesystem storage together with the database. Actually, the filesystem is backed up just after the database is dumped (using tar) and a tarfile is produced which contains both the database dump and the filesystem dump.
::yabt::mysqldb_dump_job { 'job_name':
user => 'mysql_username',
password => 'mysql_password',
hostname => 'mysql_server_host',
dbname => 'name_of_db',
storage_dir => "path_to_storage_directory",
}
Duplicity backup job
The ::yabt::duplicity_job
resource backs up a local directory to a target
location. The directory is backed up using duplicity
which must be installed
locally (note that puppet-yabt does not take care of installing rdiffbackup).
::yabt::duplicity_job { 'job_name':
source_dir => 'path_of_local_directory_to_backup',
}
Rdiffbackup backup job
The ::yabt::rdiffbackup_job
resource backs up a local directory to a target
location. The directory is backed up using rsync
which must be installed
locally (note that puppet-yabt does not take care of installing rsync,
nonetheless rsync is typically installed by default).
::yabt::rdiffbackup_job { 'job_name':
source => 'path_of_local_directory_to_backup',
}
Rsync backup job
The ::yabt::rsync_job
resource backs up a local directory to a target
location. The directory is backed up using rdiffbackup
which must be installed
locally (note that puppet-yabt does not take care of installing rdiffbackup).
::yabt::rdiffbackup_job { 'job_name':
source => 'path_of_local_directory_to_backup',
}
Status notification job
The ::yabt::status_notification_job
sends email notifications to
configured recipients to communicate the backup status or specific backup errors.
::yabt::status_notification_job
Reference
Public classes
Class: yabt
Guides the basic setup and installation of Yabt on your system.
When this class is declared with the default options, Puppet:
- Does not send any notification
- Does not configure any default destination
- Install or updates Yabt at the latest version released
You can simply declare the default Yabt class:
class { '::yabt': }
This class will download and install yabt from the distribution location. Note that dependencies are NOT installed, such as PHP or the client tools required to support the various kinds of dump or backup jobs.
Parameters within yabt
:
notifications_enabled
Enable email notifications. Default: false, which means that no email notifications are sent at all.
notifications_from
Email address of the sender of the notification email.
Default: undef, which means that no email notifications are sent at all.
notifications_recipients
Email address(es) of the recipients of the notification email.
Default: undef, which means that no default notification recipients are configured, and hence, no email notifications are sent at all.
notifications_smtp_hostname
The SMTP host to use to send email.
Default: 'localhost', which uses a locally installed SMTP server.
dump_location
The default location for dump jobs, which applies if no other dump_location
s
are set in the dump jobs.
Default: undef, which means that no default dump location is set. A dump_location
must be provided for every dump job.
duplicity_target_url
The default target URL for duplicity backup jobs.
Default: undef, which means that no default target url is set. A target_url
must be provided for every Duplicity job.
rdiffbackup_destination
Default: undef, which means that no default destination is set. A destination
must be provided for every Rdiff-backup job.
rsync_destination
Default: undef, which means that no default destination is set. A destination
must be provided for every Rdiff job.
sn_recurrence
Recurrence for the default status notification job. Use false
to disable the
default status notification job.
Default: 'weekly', which means that a weekly email is sent.
sn_at
When the default status notification job runs.
Default: undef, which means that the email will be sent at the default time (which depends on the recurrence, but is generally at 03:30).
sn_complete_recurrence
Recurrence for the backup complete status notification job. Use false
to
disable the default complete status notification job.
Default: 'monthly', which means that a monthly email with the complete status is sent.
sn_complete_at
When the default complete status notification job runs.
Default: undef, which means that the email will be sent at 03:30 of the 5th day of the month (if the default monthly recurrence is used).
ensure
Either present
or absent
. When present
then Yabt is ensured installed,
whereas absent
disinstalls Yabt.
Default: 'present'.
version
The version of Yabt to install.
Default: 'latest', which means that the latest Yabt version is installed.
Common job parameters
phase
Running phase, numeric value from 0 to 10. Jobs in a lower phase run earlier than jobs in a higher phase. Used to run notification jobs at later phases than dump and backup jobs.
Default: depends on the type of job. Dump and backup jobs generally default to phase 5, while notification jobs default to phase 9.
recurrence
Tells how often the job is run. Possible choices are: hourly
, daily
,
weekly
or monthly
. The at
parameter tells when the job will run given
its recurrence.
Default: weekly
at
Tells when the job should be run. Format differs depending on the
recurrence
value:
- for
hourly
recurrence it is in the format:MM
where MM are the minutes in the hour - for
daily
recurrence it is in the formatHH:MM
where HH:MM is the hour and minutes in the day - for
weekly
recurrence it is in the formatdayname, HH:MM
where day is the name of the day either abbreviated or full (sun, mon, tue... or sunday, monday, tuesday, ...) and HH:MM is the hour and minutes in the day - for
monthly
recurrence it is in the formatD, HH:MM
where day is the day number in the month (1, 2, ...31) and HH:MM is the hour and minutes in the day
Examples (which are also the defaults applied when the at
parameter is
omitted):
05, 03:30 ; monthly recurrence: runs on day 5 at 03:30
Sun, 03:30 ; weekly recurrence: runs on Sundays at 03:30
03:30 ; daily recurrence: runs every day at 03:30
:30 ; hourly recurrence: runs every hour at minute 30
Note that the job is not guaranteed to run at the exact time indicated in the
at
parameter. If there is another job running at the same time the job
execution will be deferred.
enabled
Default: true, which means that the job is configured to run. Use false to disable job execution, still preserving the configuration file.
ensure
Default: 'present', which means that the job is configured. Use 'absent' to remove the job.
pre_cmd
Default: none. An optional command to be executed before running the actual job. It MUST return 0 (success) otherwise the job execution is stopped. The command may do some preparatory work (e.g. stopping some services which could interfere with a clean backup) or check some pre-conditions. Example:
pre_cmd=/usr/sbin/service apache2 stop
Available since yabt version 1.2.0.
post_cmd
Default: none. An optional command to be executed after the actual job has
been run. The command may do some cleanup work (e.g. restart some services that
were stopped by pre_cmd
).
post_cmd=/usr/sbin/service apache2 start
Available since yabt version 1.2.0.
Common dump job parameters
Dump jobs produce "dump file"s out of the source content. For dump jobs, Yabt controls and takes care directly of the backup files produced, as well as retention and incrementality.
See here for additional information on dump jobs.
Dump jobs share the following parameters.
retention_period
Number of previous dumps to retain. Works together with the recurrence
parameter in the [job]
section: for daily recurrence the retention period is
a number of days while for weekly recurrence the retention is a number of weeks
Default: 14
full_period
Number of runs after which a full dump is always executed.
Default: 7
incremental
true
activates incremental dumps. Yabt will store only differences from the
previous dump, saving time and space. Not all dump jobs support incrementality.
Default: true
(where supported).
dump_location
Where dump files are stored. It is expressed in a URI-like format. It is a "root directory", each dump is stored in subdirectories of this directory. A number of possibilities are currently supported:
-
a local directory:
file:///var/backups
-
a space on a FTP server:
ftp://username:password@hostname/path
-
a space on a FTPS server (where possible this is to be preferred over FTP):
ftps://username:password@hostname/path
-
a space on a SFTP (ssh2) server:
ssh2://username:password@hostname/path
By default, the dump_location
configured with yabt
class is used.
Define: yabt::status_notification_job
Configures a status notification job. An e-mail is sent with a report of the status of the jobs.
See here for additional usage information.
Parameters within yabt::status_notification_job
:
See also the common job parameters.
complete
When true
, a complete report is produced, including also successful jobs.
When false
, only failed backups are notified and if no job failed no report
is sent at all.
Default: false
Define: yabt::dir_dump_job
Stores all files from a local directory into a .tar.bz2 file.
See here for additional usage information.
Parameters within yabt::dir_dump_job
:
See also the common job parameters and the common dump job parameters.
path
Required. The path to the directory to dump.
Class: yabt::etc_dir_dump_job
Specialized directory dump job which dumps the contents of the /etc directory.
Parameters within yabt::dir_dump_job
:
Class has no custom parameter. See the common job parameters and the common dump job parameters.
Define: yabt::mysqldb_dump_job
Dumps a MySQL database. Optionally dumps also content stored on the filesystem but referenced by the database.
See also the common job parameters and the common dump job parameters.
See here for additional usage information.
Parameters within yabt::mysqldb_dump_job
:
dbname
Required. Name of the database.
user
and password
Required. The access credentials to the database.
hostname
Location of the MySQL server. Defaults to 'localhost'
port
TCP/IP port where MySQL listens. Defaults to port 3306
storage_dir
Path to the filesystem directory where files are stored. Optional, when not provided no associated filesystem is backed up.
Define: yabt::svnrepositories_dump_job
Dumps all svn repositories contained into a directory (the "parent directory"). This job must be configured on a server which mounts the parent directory as a local directory.
See also the common job parameters and the common dump job parameters.
See here for additional usage information.
Parameters within yabt::svnrepositories_dump_job
:
parent_path
Required. The directory which contains the SVN repositories to dump. It must be a local directory.
Define: yabt::rsync_job
Back up a local directory using rsync.
See also the common job parameters.
See here for additional usage information.
Parameters within yabt::rsync_job
:
source
Path to local directory to back up.
destination
Where to back-up files to. Examples are:
user:pass@rsynchost:/path
: back up to a remote rsync server (recommended)/var/backups/
: back up to a local directory
Default: the value configured in rsync_destination
in the yabt
class.
Define: yabt::duplicity_job
Back up a local directory using duplicity.
See also the common job parameters.
See here for additional usage information.
Parameters within yabt::duplicity_job
:
source_dir
Required. Path to the local directory to back up.
passphrase
Required. A non-obvious alphanumeric string which is used by duplicity to encrypt the backup files before they are stored.
target_url
Optional. URL of the target location to backup to.
Default: the value configured in duplicity_target_url
in the yabt
class.
retention_period
Optional. Number of days to retain in the backup. Default: 180
full_period
Optional. Number of days after which to run a full backup. Default: 60
Define: yabt::rdiffbackup_job
Back up a local directory using rdiff-backup.
See here for additional usage information.
Parameters within yabt::rdiffbackup_job
:
See also the common job parameters.
source
Path to local directory to back up.
destination
The location where to store the backup copy. It can be a local destination:
/path/to/local/directory
or a destination on a remote machine:
user@hostname::/path/to/remote/directory
In this case you should pre-initialize access to the remote machine by copying the public SSH key for example as it is explained here.
Default: the value configured in rdiffbackup_destination
in the yabt
class.
retention_period
Remove the incremental backup information in the destination directory that
has been around longer than the given time. time_spec can be either an absolute
time, like "2002-01-04", or a time interval. The time interval is an integer
followed by the character s, m, h, D, W, M, or Y, indicating seconds,
minutes, hours, days, weeks, months, or years respectively, or a number of
these concatenated. For example, 32m means 32 minutes, and 3W2D10h7s means
3 weeks, 2 days, 10 hours, and 7 seconds. See also rdiff-backup's
--remove-older-than
option.
Default: '1Y' which corresponds to one year.
Limitations
This module has been developed and tested under Ubuntu Linux version 16.04 and Puppet 3.8. It was tested working under Debian Jessie. It should reasonably work under any Debian-based distribution as well as Puppet 3.8+ and 4+.
Development
Input is welcome to add support for other platforms.
Copyright
Copyright (c) Michele Manzato.
The Puppet module for Yabt is open source software licensed under the MIT License. Basically, you are free to use Yabt in any commercial or non-commercial project, you can modify the code as you wish as long as you don't change licensing and retain the original copyright statement.
See Yabt documentation for additional copyright information.
Disclaimer
This Puppet module for Yabt is released as-is. I cannot guarantee fitness of Yabt for any particular use and I cannot assume any direct or indirect liability should your system or your data be damaged or lost due to proper or improper use of Yabt, or due to bugs in the Puppet module itself or third-party programs run by this module.
2018-07-03 Michele Manzato michele.manzato@gmail.com - 1.1.0
- Added support for job options
pre_cmd
andpost_cmd
2016-06-16 Michele Manzato michele.manzato@gmail.com - 0.1.0
- Initial release under puppetlabs
Dependencies
- puppetlabs/stdlib (>= 4.11.0)
MIT License ----------- Puppet module for Yabt - Copyright (c) 2017 Michele Manzato Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.