StoRM puppet module
Table of Contents
Description
StoRM Puppet module allows administrators to easily configure StoRM services. Currently, the supported services are:
-
StoRM Backend
-
StoRM Frontend
-
StoRM WebDAV
-
StoRM Globus GridFTP server
Setup
StoRM Puppet module is available on puppet forge:
puppet module install cnafsd-storm
You can also build and install module from source code as follow:
git clone https://github.com/italiangrid/storm-puppet-module.git
cd storm-puppet-module
puppet module build
puppet module install ./pkg/cnafsd-storm-*.tar.gz
Usage
With this Puppet module, administrators can configure StoRM services. Some utility classes are also provided to configure users, storage directories and pool accounts, if needed.
StoRM users utility class
To create the default StoRM users and groups you can use the storm::users
utility class.
Use:
include storm::users
to create default scenario:
-
storm
group with id 1100 -
edguser
group with id 1101 -
storm
user with id 1100, member ofstorm
andedguser
groups -
edguser
user with id 1101, member ofedguser
andstorm
groups
You can also customize and create your own users and groups as follow:
class { 'storm::users':
groups => {
infosys => {
gid => '996',
},
},
users => {
edguser => {
comment => 'Edguser user',
groups => [ edguser, infosys, storm, ],
uid => '995',
gid => '995',
home => '/home/edguser',
},
storm => {
comment => 'StoRM user',
groups => [ storm, edguser, ],
uid => '991',
gid => '991',
home => '/home/storm',
},
},
}
Check here for all the class options.
StoRM storage utility class
To create the root directories of your storage areas, you can use the storm::storage
utility class. It's mainly used for test purposes. We expected not to use this class on production.
Use:
include storm::storage
to create /storage
directory owned by 'storm' user and '755' as permissions. You can specify a different list of directories as follow:
class { 'storm::storage':
root_directories => [
'/storage',
'/storage/test.vo',
'/storage/dteam',
],
}
Check here for all the class options.
Customize StoRM WebDAV component
Example of StoRM WebDAV configuration:
class { 'storm::webdav':
storage_areas => [
{
name => 'test.vo',
root_path => '/storage/test.vo',
access_points => ['/test.vo'],
vos => ['test.vo'],
authenticated_read_enabled => false,
anonymous_read_enabled => false,
vo_map_enabled => false,
},
{
name => 'test.vo.2',
root_path => '/storage/test.vo.2',
access_points => ['/test.vo.2'],
vos => ['test.vo.2'],
authenticated_read_enabled => true,
anonymous_read_enabled => false,
vo_map_enabled => false,
},
],
hostnames => ['storm-webdav.example.org', 'alias-for-storm-webdav.example.org'],
}
Check here for all WebDAV class options.
Customize StoRM GridFTP component
Example of StoRM Gridftp configuration:
class { 'storm::gridftp':
redirect_lcmaps_log => true,
llgt_log_file => '/var/log/storm/storm-gridftp-lcmaps.log',
}
Check here for all GridFTP class options.
Customize StoRM Frontend component
Example of StoRM Frontend configuration:
class { 'storm::frontend':
be_xmlrpc_host => 'storm-backend.example',
be_xmlrpc_token => 'secret',
db_host => 'storm-backend.example',
db_user => 'storm',
db_passwd => 'secret',
}
Check here for all Frontend class options.
Customize StoRM Backend component
StoRM Backend class:
-
installs MySQL community client/server;
-
creates and initializes StoRM database;
-
installs and configures storm-backend-server service;
-
installs and configures storm-info-provider and bdii.
| Property Name | Description | |:——————————–|:——————| | hostname
<span style=“color:red”>*</span> | StoRM Backend Fully Qualified Domain Name. Mandatory. | | install_native_libs_gpfs
| Set this if you need to install storm-native-libs-gpfs. Default value: false. | | db_username
| The name of user used to connect to local database. Default value: storm. | | db_password
| Password for the user in db_storm_username
. Default value: bluemoon. | | mysql_server_install
| | mysql_server_root_password
| | mysql_server_override_options
| | xroot_hostname
| Root server (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration. Default value: hostname
| | xroot_port
| Root server port (default value for all Storage Areas). Default value: 1094| | gsiftp_pool_balance_strategy
| | | gsiftp_pool_members
| | | webdav_pool_members
| | | srm_pool_members
| | | transfer_protocols
| | | fs_type
| | | storage_areas
| | | frontend_public_host
| | | frontend_port
| | | directory_automatic_creation
| | | directory_writeperm
| | | rest_services_port
| | | rest_services_max_threads
| | | rest_services_max_queue_size
| | | synchcall_xmlrpc_unsecure_server_port
| | | synchcall_xmlrpc_maxthread
| | | synchcall_xmlrpc_max_queue_size
| | | synchcall_xmlrpc_security_enabled
| | | synchcall_xmlrpc_security_token
| | | ptg_skip_acl_setup
| | | pinlifetime_default
| | | pinlifetime_maximum
| | | sanity_check_enabled
| | | service_du_enabled
| | | service_du_delay
| | | service_du_interval
| | | synchcall_max_ls_entries
| | | gc_pinnedfiles_cleaning_delay
| | | gc_pinnedfiles_cleaning_interval
| | | gc_purge_enabled
| | | gc_purge_interval
| | | gc_purge_size
| | | gc_expired_request_time
| | | gc_expired_inprogress_time
| | | gc_ptp_transit_interval
| | | gc_ptp_transit_start_delay
| | | extraslashes_file
| | | extraslashes_root
| | | extraslashes_gsiftp
| | | db_connection_pool_enabled
| | | db_connection_pool_max_active
| | | db_connection_pool_max_wait
| | | asynch_db_reconnect_period
| | | asynch_db_delay_period
| | | asynch_picking_initial_delay
| | | asynch_picking_time_interval
| | | asynch_picking_max_batch_size
| | | requests_scheduler_core_size
| | | requests_scheduler_max_size
| | | requests_scheduler_queue_size
| | | ptp_requests_scheduler_core_size
| | | ptp_requests_scheduler_max_size
| | | ptp_requests_scheduler_queue_size
| | | ptg_requests_scheduler_core_size
| | | ptg_requests_scheduler_max_size
| | | ptg_requests_scheduler_queue_size
| | | bol_requests_scheduler_core_size
| | | bol_requests_scheduler_max_size
| | | bol_requests_scheduler_queue_size
| | | info_sitename
| | | info_storage_default_root
| | | info_endpoint_quality_level
| | | info_webdav_pool_list
| | | info_frontend_host_list
| |
Example of StoRM Backend configuration:
class { 'storm::backend':
hostname => 'backend.test.example',
srm_pool_members => [
{
'hostname' => 'frontend.test.example',
}
],
gsiftp_pool_members => [
{
'hostname' => 'gridftp.test.example',
},
],
webdav_pool_members => [
{
'hostname' => 'webdav.test.example',
},
],
storage_areas => [
{
'name' => 'test.vo',
'root_path' => '/storage/test.vo',
'access_points' => ['/test.vo'],
'vos' => ['test.vo'],
'storage_class' => 'T0D1',
'online_size' => 4,
'transfer_protocols' => ['file', 'gsiftp', 'https'],
},
],
}
Documentation
You can find all the info about module classes and parameters at:
How to update doc
Update REFERENCE.md
file as follow:
puppet strings generate --format markdown
Update gh-pages
branch as follow:
bundle exec rake strings:gh_pages:update
Limitations
It works only on RedHat CentOS 7 distribution.