storm

StoRM Puppet module

Project URL RSS Feed Report issues

Module Stats

18,354 downloads

433 latest version

4.7 quality score

Version information

released Oct 2nd 2023

This version is compatible with:

Puppet Enterprise 2023.2.x, 2023.1.x, 2023.0.x, 2021.7.x, 2021.6.x, 2021.5.x, 2021.4.x, 2021.3.x, 2021.2.x, 2021.1.x, 2021.0.x
Puppet >= 7.0.0 < 8.0.0
,

Start using this module

Installation method

Add this module to your Puppetfile:

mod 'cnafsd-storm', '4.1.0'

Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add cnafsd-storm

Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install cnafsd-storm --version 4.1.0

Documentation

cnafsd/storm — version 4.1.0 Oct 2nd 2023

StoRM puppet module

Table of Contents:

Description
Setup
Usage
Limitations - OS compatibility

Description

StoRM Puppet module allows administrators to configure StoRM services deployed on CentOS 7.

The supported services are:

StoRM Backend
StoRM Frontend
StoRM WebDAV
StoRM Globus GridFTP

Setup

StoRM Puppet module is available on puppet forge:

puppet module install cnafsd-storm

You can also build and install module from source code as follow:

git clone https://github.com/italiangrid/storm-puppet-module.git
cd storm-puppet-module
pdk build
puppet module install ./pkg/cnafsd-storm-*.tar.gz

Usage

This Puppet module allows site administrators to properly configure StoRM services on CentOS 7 platform. This module provides some classes related to the main components and also some utility classes those can be used to configure StoRM repositories, StoRM users, VO pool accounts, LCMAPS and storage directories.

Component classes:

StoRM Backend class
StoRM Frontend class
StoRM WebDAV class
StoRM GridFTP class

Utility classes:

StoRM database class
StoRM repo class
StoRM users class

StoRM Backend class

Prerequisites: A MySQL or MariaDB server with StoRM databases must exist. Databases can be empty. If you want to use this module to install MySQL client and server and init databases, please read about StoRM database utility class.

The Backend class installs storm-backend-server and all its related packages, such as storm-native-libs, storm-native-libs-gpfs in case GPFS is used as filesystem and storm-dynamic-info-provider.

The Backend class configures storm-backend-server service by managing the following files:

/etc/storm/backend-server/storm.properties
/etc/storm/backend-server/namespace.xml
/etc/systemd/system/storm-backend-server.service.d/storm-backend-server.conf
/etc/systemd/system/storm-backend-server.service.d/filelimit.conf

and deploys StoRM databases. In addiction, this class configures and run storm-info-provider by managing the following file:

/etc/storm/info-provider/storm-yaim-variables.conf.

The whole list of StoRM Backend class parameters can be found here.

Example of StoRM Backend configuration:

class { 'storm::backend':
  db_password           => 'secret-password',
  transfer_protocols    => ['file', 'gsiftp', 'webdav'],
  xmlrpc_security_token => 'secret-token',
  srm_pool_members      => [
    {
      'hostname' => 'frontend-public-host.example',
    },
    {
      'hostname' => 'other-frontend-host.example',
    }
  ],
  gsiftp_pool_members   => [
    {
      'hostname' => 'gridftp-01.example',
    },
    {
      'hostname' => 'gridftp-02.example',
    }
  ],
  webdav_pool_members   => [
    {
      'hostname' => webdav-01.example,
    },
    {
      'hostname' => webdav-02.example,
    }
  ],
  storage_areas         => [
    {
      'name'          => 'dteam-disk',
      'root_path'     => '/storage/disk',
      'access_points' => ['/disk'],
      'vos'           => ['dteam'],
      'online_size'   => 40,
    },
    {
      'name'          => 'dteam-tape',
      'root_path'     => '/storage/tape',
      'access_points' => ['/tape'],
      'vos'           => ['dteam'],
      'online_size'   => 40,
      'nearline_size' => 80,
      'fs_type'       => 'gpfs',
      'storage_class' => 'T1D0',
    },
  ],
}

Starting from Puppet module v2.0.0, the management of Storage Site Report has been improved. Site administrators can add script and cron described in the how-to using a defined type storm::backend::storage_site_report. For example:

storm::backend::storage_site_report { 'storage-site-report':
  report_path => '/storage/info/report.json', # the internal storage area path
  minute      => '*/20', # set cron's minute
}

Enable HTTP as transfer protocol for SRM

To enable HTTP as transfer protocol for SRM prepare-to-get and prepare-to-put requests, you must add webdav protocol to the list of your transfer_protocols and define at least one member for webdav_pool_members. You can re-define the default list of transfer protocols by adding your storm::backend::transfer_protocols variable and/or you can override this list by adding a specific transfer_protocols for each storage area:

class { 'storm::backend':
  # ...
  'webdav_pool_members' => [
    {
      'hostname' => webdav.test.example,
    },
  ],
  # defines the default list of transfer protocols for each storage area:
  'transfer_protocols'  => ['file', 'gsiftp', 'webdav'], 
  'storage_areas'       => [
    {
      'name'          => 'sa-http-enabled',
      'root_path'     => '/storage/sa-http-enabled',
      'access_points' => ['/sa-http-enabled'],
      'vos'           => ['test.vo'],
      'online_size'   => 40,
    },
    {
      'name'               => 'sa-no-http-enabled',
      'root_path'          => '/storage/sa-no-http-enabled',
      'access_points'      => ['/sa-no-http-enabled'],
      'vos'                => ['test.vo'],
      'online_size'        => 40,
      # disable webdav protocol for this storage area
      'transfer_protocols' => ['file', 'gsiftp'],
    },
    # ...
  ],
  # ...
}

The manifest.pp showed above includes the HTTP transfer protocol for all the storage area defined. By default, storm::backend::transfer_protocols includes only file and gsiftp.

StoRM Frontend class

The StoRM Frontend class installs storm-frontend-mp and all the releated packages and configures storm-frontend-server service by managing the following files:

/etc/storm/frontend-server/storm-frontend-server.conf
/etc/sysconfig/storm-frontend-server

The whole list of StoRM Frontend class parameters can be found here.

Example of StoRM Frontend configuration:

class { 'storm::frontend':
  be_xmlrpc_host  => 'backend.test.example',
  be_xmlrpc_token => 'NS4kYAZuR65XJCq',
  db_host         => 'backend.test.example',
  db_user         => 'storm',
  db_passwd       => 'secret-password',
}

StoRM WebDAV class

The StoRM WebDAV class installs storm-webdav rpm and configures storm-webdav service by managing the following files:

the systemd override files filelimit.conf and storm-webdav.conf stored into /etc/systemd/system/storm-webdav.service.d;
the storage areas property files stored into /etc/storm/webdav/sa.d (optional);

The whole list of StoRM Webdav class parameters can be found here.

Example of StoRM WebDAV configuration:

class { 'storm::webdav':
  storage_areas => [
    {
      name                       => 'test.vo',
      root_path                  => '/storage/test.vo',
      access_points              => ['/test.vo'],
      vos                        => ['test.vo'],
    },
    {
      name                       => 'test.vo.2',
      root_path                  => '/storage/test.vo.2',
      access_points              => ['/test.vo.2', '/alias'],
      vos                        => ['test.vo.2'],
      authenticated_read_enabled => true,
    },
  ],
  hostnames => ['storm-webdav.test.example', 'alias-for-storm-webdav.test.example'],
}

Storage Areas can also be configured singularly by using the defined type storm::webdav::storage_area_file. This strategy allows site administrators to keep their manifests unaware of the improvements on StoRM WebDAV code. For example, if a new property is added into Storage Area configuration files, you haven't to update your Puppet module and all the service configuration will continue working.

Example of Storage Areas configuration done with storm::webdav::storage_area_file:

class { 'storm::webdav':
  hostnames => ['storm-webdav.test.example', 'alias-for-storm-webdav.test.example'],
}

storm::webdav::storage_area_file { 'test.vo.properties':
  source => '/path/to/my/test.vo.properties',
}

storm::webdav::storage_area_file { 'test.vo.2.properties':
  source => '/path/to/my/test.vo.2.properties',
}

Starting from Puppet module v2.0.0, the management of application.yml file has been removed from storm::webdav class. Site administrators can edit their own configuration files or use a defined type storm::webdav::application_file to inject also one or more YAML files into the proper directory. For example:

class { 'storm::webdav':
  jvm_opts => '-Xms1024m -Xmx1024m -Dspring.profiles.active=extra',
}

storm::webdav::application_file { 'application.yml':
  source => '/path/to/my/application.yml',
}

storm::webdav::application_file { 'application-extra.yml':
  source => '/path/to/my/application-extra.yml',
}

StoRM GridFTP class

The StoRM GridFTP class installs storm-globus-gridftp-mp and configures storm-globus-gridftp service by managing the following files:

/etc/grid-security/gridftp.conf, the main configuration file;
/etc/sysconfig/storm-globus-gridftp, with the environment variables.

The whole list of StoRM GridFTP class parameters can be found here.

Examples of StoRM Gridftp configuration:

class { 'storm::gridftp':
  redirect_lcmaps_log => true,
  llgt_log_file       => '/var/log/storm/storm-gridftp-lcmaps.log',
}

StoRM database class

The StoRM database utility class installs mariadb server and releated rpms and configures mysql service by managing the following files:

/etc/my.cnf.d/server.cnf;
/etc/systemd/system/mariadb.service.d/limits.conf.

The whole list of StoRM Database class parameters can be found here.

Examples of StoRM Database usage:

class { 'storm::db':
  root_password => 'supersupersecretword',
  storm_password => 'supersecretword',
}

StoRM repo class

The StoRM repo utility class creates all the StoRM YUM repositories: stable, beta, nightly. By default, only stable repo is enabled. You can also add extra repositories to install.

The whole list of StoRM repo class parameters can be found here.

Examples of StoRM Repo usage:

class { 'storm::repo':
  enabled => ['stable', 'beta'],
}

StoRM users class

The StoRM users utility class creates the default StoRM users and groups.

Use:

include storm::users

to create default scenario:

storm group with id 1100
edguser group with id 1101
storm user with id 1100, member of storm and edguser groups
edguser user with id 1101, member of edguser and storm groups

You can also customize and create your own users and groups as follow:

class { 'storm::users':
  groups => {
    infosys => {
      gid => '996',
    },
  },
  users  => {
    edguser => {
      comment => 'Edguser user',
      groups  => [ edguser, infosys, storm, ],
      uid     => '995',
      gid     => '995',
      home    => '/home/edguser',
    },
    storm   => {
      comment => 'StoRM user',
      groups  => [ storm, edguser, ],
      uid     => '991',
      gid     => '991',
      home    => '/home/storm',
    },
  },
}

The whole list of StoRM repo class parameters can be found here.

Documentation

You can find all the info about module classes and parameters at:

Developers

Run tests with:

pdk test unit

Validate code with:

pdk validate

Limitations

It works only on RedHat CentOS 7 distribution.

Reference

Classes

storm::backend: StoRM Backend puppet module
storm::backend::config: StoRM Backend config class
storm::backend::configdb: StoRM Backend DB config class
storm::backend::install: StoRM Backend install class
storm::backend::service: StoRM Backend service class
storm::db: Install MariaDB server and client, create empty databases 'storm_db' and 'storm_be_ISAM', add storm user and all the necessary grants.
storm::frontend: StoRM Frontend puppet module
storm::frontend::config: StoRM Frontend config class
storm::frontend::install: StoRM Frontend install class
storm::frontend::service: StoRM Frontend service class
storm::gridftp: StoRM GridFTP puppet module
storm::gridftp::config: StoRM GridFTP config class
storm::gridftp::install: StoRM GridFTP install class
storm::gridftp::service: StoRM GridFTP service class
storm::repo: Choose which StoRM repository you want to intall and enable. Also a custom list of repository URL can be specified.
storm::users: StoRM accounts configuration
storm::webdav: StoRM WebDAV puppet module
storm::webdav::config: StoRM WebDAV config class
storm::webdav::install: StoRM WebDAV install class
storm::webdav::service: StoRM WebDAV service class

Defined types

storm::backend::service_conf_file: Starting from Puppet module v2.0.0, site administrators can inject one or more .conf files into /etc/systemd/system/storm-backend-server.service.d directory.
storm::backend::storage_site_report: Starting from Puppet module v2.0.0, the management of Storage Site Report has been improved. Site administrators can add script and cron described in the how-to
storm::rootdir: StoRM main storage area root directory defined resource
storm::sarootdir: StoRM Storage Area root directory defined resource
storm::webdav::application_file: Starting from Puppet module v2.0.0, the management of application.yml file has been removed from storm::webdav class. Site administrators can edit their own configuration files or use this defined type to inject one or more YAML files into the proper directory.
storm::webdav::drop_in_file: Use this define to inject one or more .conf files into /etc/systemd/system/storm-webdav.service.d directory.
storm::webdav::storage_area_file: Storage Areas can be configured singularly by using this defined type. This strategy allows site administrators to keep their manifests unaware of the improvements on StoRM WebDAV code. For example, if a new property is added into Storage Area configuration files, you haven't to update your Puppet module and all the service configuration will continue working.

Data types

Storm::Backend::Acl: The ACL type for storm-backend-server
Storm::Backend::AclMode: The AclMode type for storm-backend-server
Storm::Backend::BalanceStrategy: The BalanceStrategy type for storm-backend-server
Storm::Backend::Endpoint: The Endpoint type for storm-backend-server
Storm::Backend::FileSystem: The FileSystem type for storm-backend-server
Storm::Backend::FsDriver: The FsDriver type for storm-backend-server
Storm::Backend::FsType: The FsType type for storm-backend-server
Storm::Backend::Gsiftp: The Gsiftp type for storm-backend-server
Storm::Backend::GsiftpPoolMember: The GsiftpPoolMember type for storm-backend-server
Storm::Backend::Pool: The Pool type for storm-backend-server
Storm::Backend::Quota: The Quota type for storm-backend-server
Storm::Backend::Rfio: The Rfio type for storm-backend-server
Storm::Backend::SrmPoolMember: The SrmPoolMember type for storm-backend-server
Storm::Backend::StorageArea: The storage area type for storm-backend-server
Storm::Backend::StorageClass: The StorageClass type for storm-backend-server
Storm::Backend::TransferProtocol: The TransferProtocol type for storm-backend-server
Storm::Backend::Webdav: The WebDAV type for storm-backend-server
Storm::Backend::WebdavPoolMember: The WebdavPoolMember type for storm-backend-server
Storm::Backend::Xroot: The Xroot type for storm-backend-server
Storm::CustomRepo: The storage area type for storm-webdav
Storm::Webdav::StorageArea: The storage area type for storm-webdav.

Classes

`storm::backend`

StoRM Backend puppet module

Examples

Example of usage

class { 'storm::backend':
  hostname => 'backend.test.example',
  db_username => 'storm',
  db_password => 'bluemoon',
  srm_pool_members => [
    {
      'hostname' => 'frontend.test.example',
    }
  ],
  gsiftp_pool_members => [
    {
      'hostname' => 'gridftp.test.example',
    },
  ],
  webdav_pool_members => [
    {
      'hostname' => 'webdav.test.example',
    },
  ],
  storage_areas => [
    {
      'name'               => 'test.vo',
      'root_path'          => '/storage/test.vo',
      'access_points'      => ['/test.vo'],
      'vos'                => ['test.vo'],
      'storage_class'      => 'T0D1',
      'online_size'        => 4,
      'transfer_protocols' => ['file', 'gsiftp', 'https'],
    },
  ],
}

Parameters

The following parameters are available in the storm::backend class:

hostname
db_hostname
db_username
db_password
xroot_hostname
xroot_port
gsiftp_pool_balance_strategy
gsiftp_pool_members
webdav_pool_balance_strategy
webdav_pool_members
srm_pool_members
transfer_protocols
fs_type
storage_areas
frontend_public_host
frontend_port
directory_automatic_creation
directory_writeperm
rest_services_port
rest_services_max_threads
rest_services_max_queue_size
xmlrpc_unsecure_server_port
xmlrpc_maxthread
xmlrpc_max_queue_size
xmlrpc_security_enabled
xmlrpc_security_token
ptg_skip_acl_setup
pinlifetime_default
pinlifetime_maximum
sanity_check_enabled
service_du_enabled
service_du_delay
service_du_interval
max_ls_entries
gc_pinnedfiles_cleaning_delay
gc_pinnedfiles_cleaning_interval
gc_purge_enabled
gc_purge_interval
gc_purge_size
gc_expired_request_time
gc_expired_inprogress_time
gc_ptp_transit_interval
gc_ptp_transit_start_delay
extraslashes_file
extraslashes_root
extraslashes_gsiftp
db_connection_pool_enabled
db_connection_pool_max_active
db_connection_pool_max_wait
asynch_db_reconnect_period
asynch_db_delay_period
asynch_picking_initial_delay
asynch_picking_time_interval
asynch_picking_max_batch_size
requests_scheduler_core_size
requests_scheduler_max_size
requests_scheduler_queue_size
ptp_requests_scheduler_core_size
ptp_requests_scheduler_max_size
ptp_requests_scheduler_queue_size
ptg_requests_scheduler_core_size
ptg_requests_scheduler_max_size
ptg_requests_scheduler_queue_size
bol_requests_scheduler_core_size
bol_requests_scheduler_max_size
bol_requests_scheduler_queue_size
info_config_file
info_sitename
info_storage_default_root
info_endpoint_quality_level
info_webdav_pool_list
info_frontend_host_list
jvm_options
jmx
jmx_options
lcmaps_db_file
lcmaps_policy_name
lcmaps_log_file
lcmaps_debug_level
http_turl_prefix
storm_limit_nofile
manage_path_authz_db
path_authz_db_file
manage_storm_properties
path_storm_properties

`hostname`

Data type: String

StoRM Backend Fully Qualified Domain Name

Default value: $fqdn

`db_hostname`

Data type: String

Fully Qualified Domain Name of database hostname. Default value: hostname.

Default value: $hostname

`db_username`

Data type: String

The name of user used to connect to local database. Default: storm

`db_password`

Data type: String

Password for the user in db_storm_username. Default: bluemoon

`xroot_hostname`

Data type: String

Root server (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

Default value: $hostname

`xroot_port`

Data type: Integer

Root server port (default value for all Storage Areas).

`gsiftp_pool_balance_strategy`

Data type: Storm::Backend::BalanceStrategy

Load balancing strategy for GridFTP server pool (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration. Available values: round-robin, smart-rr, random, weight. Default value: round-robin. See Storm::Backend::BalanceStrategy.

`gsiftp_pool_members`

Data type: Array[Storm::Backend::GsiftpPoolMember]

Array of Storm::Backend::GsiftpPoolMember. GridFTP servers pool list (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

`webdav_pool_balance_strategy`

Data type: Storm::Backend::BalanceStrategy

Load balancing strategy for WebDAV server pools (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration. Available values: round-robin, smart-rr, random, weight. Default value: round-robin. See Storm::Backend::BalanceStrategy.

`webdav_pool_members`

Data type: Array[Storm::Backend::WebdavPoolMember]

Array of Storm::Backend::WebdavPoolMember. WebDAV endpoints pool list (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

`srm_pool_members`

Data type: Array[Storm::Backend::SrmPoolMember]

Array of Storm::Backend::SrmPoolMember. Frontend endpoints pool list (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

`transfer_protocols`

Data type: Array[Storm::Backend::TransferProtocol]

List of supported (and published) transfer protocols (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

`fs_type`

Data type: Storm::Backend::FsType

File System Type (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration. Available values: posixfs, gpfs and test. Default value: posixfs See Storm::Backend::FsType.

`storage_areas`

Data type: Array[Storm::Backend::StorageArea]

List of supported Storage Areas. Array of Storm::Backend::StorageArea.

`frontend_public_host`

Data type: String

StoRM Frontend service public host. It’s used by StoRM Info Provider to publish the SRM endpoint into the Resource BDII. Default value: hostname

Default value: $hostname

`frontend_port`

Data type: Integer

StoRM Frontend service port. Default value: 8444

`directory_automatic_creation`

Data type: Boolean

Flag to enable automatic missing directory creation upon srmPrepareToPut requests. Default: false.

`directory_writeperm`

Data type: Boolean

Flag to enable directory write permission setting upon srmMkDir requests on created directories. Default: false.

`rest_services_port`

Data type: Integer

REST services port. Default value: 9998

`rest_services_max_threads`

Data type: Integer

REST services max active requests. Default: 100

`rest_services_max_queue_size`

Data type: Integer

REST services max queue size of accepted requests. Default: 1000

`xmlrpc_unsecure_server_port`

Data type: Integer

Port to listen on for incoming XML-RPC connections from Frontends(s). Default: 8080

`xmlrpc_maxthread`

Data type: Integer

Number of threads managing XML-RPC connection from Frontends(s). A well sized value for this parameter have to be at least equal to the sum of the number of working threads in all Frontends. Default: 100.

`xmlrpc_max_queue_size`

Data type: Integer

Max number of accepted and queued XML-RPC connection from Frontends(s). Default: 1000

`xmlrpc_security_enabled`

Data type: Boolean

Whether the backend will require a token to be present for accepting XML-RPC requests. Default: true.

`xmlrpc_security_token`

Data type: String

The token that the backend will require to be present for accepting XML-RPC requests. Mandatory if xmlrpc_security_enabled is true.

`ptg_skip_acl_setup`

Data type: Boolean

Skip ACL setup for PtG requests. Default: false.

`pinlifetime_default`

Data type: Integer

Default PinLifetime in seconds used for pinning files in case of srmPrepareToPut or srmPrepareToGet operation without any pinLifetime specified. Default: 259200.

`pinlifetime_maximum`

Data type: Integer

Maximum PinLifetime allowed in seconds. Default: 1814400.

`sanity_check_enabled`

Data type: Boolean

Enable/disable sanity checks during bootstrap phase. Default: true.

`service_du_enabled`

Data type: Boolean

Flag to enable disk usage service. Default: false.

`service_du_delay`

Data type: Integer

The initial delay before the service is started (seconds). Default: 60.

`service_du_interval`

Data type: Integer

The interval in seconds between successive run. Default: 360.

`max_ls_entries`

Data type: Integer

Maximum number of entries returned by an srmLs call. Since in case of recursive srmLs results can be in order of million, this prevent a server overload. Default: 500.

`gc_pinnedfiles_cleaning_delay`

Data type: Integer

Initial delay before starting the reserved space, JIT ACLs and pinned files garbage collection process, in seconds. Default: 10.

`gc_pinnedfiles_cleaning_interval`

Data type: Integer

Time interval in seconds between successive purging run. Default: 300.

`gc_purge_enabled`

Data type: Boolean

Enable the request garbage collector. Default: true.

`gc_purge_interval`

Data type: Integer

Time interval in seconds between successive purging run. Default: 600.

`gc_purge_size`

Data type: Integer

Number of requests picked up for cleaning from the requests garbage collector at each run. This value is use also by Tape Recall Garbage Collector. Default: 800

`gc_expired_request_time`

Data type: Integer

Time in seconds to consider a request expired after its submission. Default: 604800 seconds (1 week). From StoRM 1.11.13 it is used also to identify how much time is needed to consider a completed recall task as cleanable.

`gc_expired_inprogress_time`

Data type: Integer

Time in seconds to consider an in-progress ptp request as expired. Default: 2592000 seconds (1 month).

`gc_ptp_transit_interval`

Data type: Integer

Time interval in seconds between successive expired put requests agent run. Default: 3000.

`gc_ptp_transit_start_delay`

Data type: Integer

Initial delay before starting the expired put requests agent process, in seconds. Default: 60

`extraslashes_file`

Data type: String

Add extra slashes after the “authority” part of a TURL for file protocol. Defaul: ''

`extraslashes_root`

Data type: String

Add extra slashes after the “authority” part of a TURL for xroot protocol. Default: '/'

`extraslashes_gsiftp`

Data type: String

Add extra slashes after the “authority” part of a TURL for gsiftp protocol. Default: '/'

`db_connection_pool_enabled`

Data type: Boolean

Enable the database connection pool. Default: true

`db_connection_pool_max_active`

Data type: Integer

Database connection pool max active connections. Default: 10

`db_connection_pool_max_wait`

Data type: Integer

Database connection pool max wait time to provide a connection. Default: 50

`asynch_db_reconnect_period`

Data type: Integer

Database connection refresh time intervall in seconds. Default: 18000

`asynch_db_delay_period`

Data type: Integer

Database connection refresh initial delay in seconds. Default: 30.

`asynch_picking_initial_delay`

Data type: Integer

Initial delay before starting to pick requests from the DB, in seconds. Default: 1.

`asynch_picking_time_interval`

Data type: Integer

Polling interval in seconds to pick up new SRM requests. Default: 2.

`asynch_picking_max_batch_size`

Data type: Integer

Maximum number of requests picked up at each polling time. Default: 100.

`requests_scheduler_core_size`

Data type: Integer

Crusher Scheduler worker pool base size. Default: 50.

`requests_scheduler_max_size`

Data type: Integer

Crusher Schedule worker pool max size. Default: 200.

`requests_scheduler_queue_size`

Data type: Integer

Request queue maximum size. Default: 2000.

`ptp_requests_scheduler_core_size`

Data type: Integer

PrepareToPut worker pool base size. Default: 50.

`ptp_requests_scheduler_max_size`

Data type: Integer

PrepareToPut worker pool max size. Default: 200.

`ptp_requests_scheduler_queue_size`

Data type: Integer

PrepareToPut request queue maximum size. Default: 1000.

`ptg_requests_scheduler_core_size`

Data type: Integer

PrepareToGet worker pool base size. Default: 50.

`ptg_requests_scheduler_max_size`

Data type: Integer

PrepareToGet worker pool max size. Default: 200.

`ptg_requests_scheduler_queue_size`

Data type: Integer

PrepareToGet request queue maximum size. Default: 2000.

`bol_requests_scheduler_core_size`

Data type: Integer

BringOnline worker pool base size. Default: 50.

`bol_requests_scheduler_max_size`

Data type: Integer

BringOnline Worker pool max size. Default: 200.

`bol_requests_scheduler_queue_size`

Data type: Integer

BringOnline request queue maximum size. Default: 2000.

`info_config_file`

Data type: String

The key-value file used by StoRM Info Provider as its configuration file.

`info_sitename`

Data type: String

It’s the human-readable name of your site used to set the Glue-SiteName attribute.

`info_storage_default_root`

Data type: String

Default directory for Storage Areas.

`info_endpoint_quality_level`

Data type: Integer

Endpoint maturity level to be published by the StoRM info provider. Default value: 2.

`info_webdav_pool_list`

Data type: Array[Storm::Backend::WebdavPoolMember]

List of published webdav endpoints.

`info_frontend_host_list`

Data type: Array[Storm::Backend::SrmPoolMember]

List of published srm endpoints.

`jvm_options`

Data type: String

`jmx`

Data type: Boolean

`jmx_options`

Data type: String

`lcmaps_db_file`

Data type: String

`lcmaps_policy_name`

Data type: String

`lcmaps_log_file`

Data type: String

`lcmaps_debug_level`

Data type: Integer

`http_turl_prefix`

Data type: String

`storm_limit_nofile`

Data type: Integer

Sets LimitNOFILE value.

`manage_path_authz_db`

Data type: Boolean

If true, allows to set content of path-authz.db file.

`path_authz_db_file`

Data type: String

If manage_path_authz_db is true, set the content from this source path

`manage_storm_properties`

Data type: Boolean

If true, allows to set content of storm.properties file wuth the file pointed by path_storm_properties.

`path_storm_properties`

Data type: String

If manage_storm_properties is true, set the content of storm.properties from this source path

`storm::backend::config`

StoRM Backend config class

`storm::backend::configdb`

StoRM Backend DB config class

`storm::backend::install`

StoRM Backend install class

`storm::backend::service`

StoRM Backend service class

`storm::db`

Install MariaDB server and client, create empty databases 'storm_db' and 'storm_be_ISAM', add storm user and all the necessary grants.

Examples

Basic usage:

include storm::db

Parameters

The following parameters are available in the storm::db class:

fqdn_hostname
root_password
storm_username
storm_password
override_options
limit_no_file

`fqdn_hostname`

Data type: String

The Fully Qualified Domain Name of the host. Default value got from Puppet fact fqdn.

Default value: $fqdn

`root_password`

Data type: String

MySQL root password. Default: 'storm'.

`storm_username`

Data type: String

The username of the user used by storm services to query the databases. Default 'storm'.

`storm_password`

Data type: String

The password of 'storm' username used by storm services to access the databases. Default: 'storm'.

`override_options`

Data type: Data

MySQL server override options. Read more about this at https://forge.puppet.com/puppetlabs/mysql/reference#override_options.

`limit_no_file`

Data type: Integer

MariaDB setting for limitNoFile

`storm::frontend`

StoRM Frontend puppet module

Examples

Example of usage

class { 'storm::frontend':
  be_xmlrpc_host  => 'storm-backend.host.org',
  be_xmlrpc_token => 'my-secret-xmlrpc-token',
  db_host         => 'storm-backend.host.org',
  db_passwd       => 'my-secret-db-password',
}

Parameters

The following parameters are available in the storm::frontend class:

storm_frontend_server_conf_file
be_xmlrpc_host
be_xmlrpc_port
be_xmlrpc_token
be_xmlrpc_path
be_recalltable_port
db_host
db_user
db_passwd
port
threadpool_maxpending
threadpool_threads_number
gsoap_maxpending
check_user_blacklisting
argus_pepd_endpoint
argus_resource_id
monitoring_enabled
monitoring_time_interval
monitoring_detailed
security_enable_vomscheck
log_debuglevel

`storm_frontend_server_conf_file`

Data type: String

Use this parameter to provide your own storm-frontend-server.conf file. This can be used, for example, when you're using the latest module version to configure an old StoRM deployment. This parameter is used only when it has a non-empty value and it overwrites several other module parameters. Default value: empty string (that means not used).

`be_xmlrpc_host`

Data type: String

StoRM Backend hostname.

Default value: $fqdn

`be_xmlrpc_port`

Data type: Integer

StoRM Backend XML-RPC server port. Default value: 8080.

`be_xmlrpc_token`

Data type: String

Security token used for communicating with StoRM Backend. Mandatory.

`be_xmlrpc_path`

Data type: String

StoRM Backend XML-RPC server path. Default value: '/RPC2'.

`be_recalltable_port`

Data type: Integer

StoRM Backend REST server port running on the Backend machine. Default value: 9998.

`db_host`

Data type: String

Host for database connection.

Default value: $be_xmlrpc_host

`db_user`

Data type: String

User for database connection. Default value: 'storm'.

`db_passwd`

Data type: String

Password for database connection. Default value: 'storm'.

`port`

Data type: Integer

Frontend service port. Default value: 8444.

`threadpool_maxpending`

Data type: Integer

Size of the internal queue used to maintain SRM tasks in case there are no free worker threads. Default value: 200.

`threadpool_threads_number`

Data type: Integer

Size of the worker thread pool. Default value: 50.

`gsoap_maxpending`

Data type: Integer

Size of the GSOAP queue used to maintain pending SRM requests. Default value: 1000.

`check_user_blacklisting`

Data type: Boolean

Enable/disable user blacklisting. Default value: false.

`argus_pepd_endpoint`

Data type: String

The complete service endpoint of Argus PEP server. Mandatory if check_user_blacklisting is true.

`argus_resource_id`

Data type: String

The resource id is used to target a resource (or set of resources, if wildcards are used) under the control of Argus authorization. Mandatory if check_user_blacklisting is true.

`monitoring_enabled`

Data type: Boolean

Enable/disable monitoring. Default value: true.

`monitoring_time_interval`

Data type: Integer

Time interval in seconds between each monitoring round. Default value: 60.

`monitoring_detailed`

Data type: Boolean

Enable/disable detailed monitoring. Default value: false.

`security_enable_vomscheck`

Data type: Boolean

Flag to enable/disable checking proxy VOMS credentials. Default value: true.

`log_debuglevel`

Data type: String

Logging level. Possible values are: ERROR, WARN, INFO, DEBUG, DEBUG2. Default value: INFO.

`storm::frontend::config`

StoRM Frontend config class

`storm::frontend::install`

StoRM Frontend install class

`storm::frontend::service`

StoRM Frontend service class

`storm::gridftp`

Parameters

The StoRM GridFTP configuration parameters are:

port: the port used by GridFTP server service;
port_range: the range of ports used by transfer sockets; format is 'MIN,MAX';
connections_max: the number of max allowed connections to server;

Examples

Example of usage

class { 'storm::gridftp':
  port            => 2811,
  port_range      => '20000,25000',
  connections_max => 2000,
}

Parameters

The following parameters are available in the storm::gridftp class:

port
port_range
connections_max
log_single
log_transfer
redirect_lcmaps_log
llgt_log_file
lcmaps_debug_level
lcas_debug_level
load_storm_dsi_module
log_level
data_interface

`port`

Data type: Integer

The port used by GridFTP server service.

`port_range`

Data type: String

The range of ports used by transfer sockets; format is 'MIN,MAX'.

`connections_max`

Data type: Integer

The number of max allowed connections to server.

`log_single`

Data type: String

Session log file path. Default is: /var/log/storm/storm-gridftp-session.log

`log_transfer`

Data type: String

Transfer log file path. Default is: /var/log/storm/storm-globus-gridftp.log

`redirect_lcmaps_log`

Data type: Boolean

If true, redirect the LCMAPS log to the file specified by 'llgt_log_file'.

`llgt_log_file`

Data type: String

The LCMAPS log file used if 'redirect_lcmaps_log' is true.

`lcmaps_debug_level`

Data type: Integer

The LCMAPS logging level. Values from 0 (ERROR) to 5 (DEBUG). Default: 3 (INFO)

`lcas_debug_level`

Data type: Integer

The LCAS logging level. Values from 0 (ERROR) to 5 (DEBUG). Default: 3 (INFO)

`load_storm_dsi_module`

Data type: Boolean

Enable/Disable StoRM DSI module. Default: true (enabled)

`log_level`

Data type: String

`data_interface`

Data type: Optional[String]

(Optional) Hostname or IP address of the interface to use for data connections. Set this to your external IP address in case your public address is different from your local address (e.g. the floating IP address of a Virtual Machine).

`storm::gridftp::config`

StoRM GridFTP config class

`storm::gridftp::install`

StoRM GridFTP install class

`storm::gridftp::service`

StoRM GridFTP service class

`storm::repo`

Choose which StoRM repository you want to intall and enable. Also a custom list of repository URL can be specified.

Examples

Install all the repositories and enable only nightly repo as follow:

class { 'storm::repo':
  enabled => ['stable'],
}

Parameters

The following parameters are available in the storm::repo class:

installed
enabled
extra

`installed`

Data type: Array[Enum['stable', 'beta', 'nightly']]

The list of repositories that have to be installed. Allowed values are stable, beta and nightly. Optional.

`enabled`

Data type: Array[Enum['stable', 'beta', 'nightly']]

The list of repositories that have to be enabled. Allowed values are stable, beta and nightly. Optional.

`extra`

Data type: Array[Storm::CustomRepo]

A list of repository that have to be created. Optional.

`storm::users`

Parameters

StoRM needs a 'storm' Unix user, member of an 'edguser' group. This class creates all the necessary users.

Examples

Example of usage

class { 'storm::users':
  groups => {
    'infosys' => {
      gid => '996',
    },
  },
  users => {
    'edguser' => {
      'comment' => 'Edguser user',
      'groups'  => [ edguser, infosys, storm, ],
      'uid'     => '995',
      'gid'     => '995',
    },
    'storm' => {
      'comment' => 'StoRM user',
      'groups'  => [ storm, edguser, ],
      'uid'     => '991',
      'gid'     => '991',
    },
  }
}

Parameters

The following parameters are available in the storm::users class:

groups
users

`groups`

Data type: Accounts::Group::Hash

`users`

Data type: Accounts::User::Hash

`storm::webdav`

StoRM WebDAV puppet module

Examples

Basic usage


class { 'storm::webdav':
  storage_areas => [
    {
      name                       => 'test.vo',
      root_path                  => '/storage/test.vo',
    },
  ],
}

Parameters

The following parameters are available in the storm::webdav class:

storage_areas
hostnames
http_port
https_port
trust_anchors_refresh_interval
max_concurrent_connections
max_queue_size
connector_max_idle_time
vo_map_files_enable
vo_map_files_config_dir
vo_map_files_refresh_interval
tpc_max_connections
tpc_max_connections_per_route
tpc_verify_checksum
tpc_timeout_in_secs
tpc_tls_protocol
tpc_report_delay_secs
tpc_enable_tls_client_auth
tpc_progress_report_thread_pool_size
tpc_enable_expect_continue_threshold
jvm_opts
authz_server_enable
authz_server_issuer
authz_server_max_token_lifetime_sec
authz_server_secret
require_client_cert
use_conscrypt
tpc_use_conscrypt
enable_http2
debug
debug_port
debug_suspend
storm_limit_nofile
user
tape_wellknown_source
tls_protocol
checksum_filter_enabled
macaroon_filter_enabled
file_buffer_size
voms_trust_store_dir
voms_trust_store_refresh_interval_sec
voms_cache_enabled
voms_cache_entry_lifetime_sec

`storage_areas`

Data type: Array[Storm::Webdav::StorageArea]

List of storage area's configuration. Default value: empty list.

`hostnames`

Data type: Array[String]

Sets STORMWEBDAV_HOSTNAME(N) environment variables.

`http_port`

Data type: Integer

Sets STORM_WEBDAV_HTTP_PORT environment variable.

`https_port`

Data type: Integer

Sets STORM_WEBDAV_HTTPS_PORT environment variable.

`trust_anchors_refresh_interval`

Data type: Integer

Sets STORM_WEBDAV_TRUST_ANCHORS_REFRESH_INTERVAL environment variable.

`max_concurrent_connections`

Data type: Integer

Sets STORM_WEBDAV_MAX_CONNECTIONS environment variable.

`max_queue_size`

Data type: Integer

Sets STORM_WEBDAV_MAX_QUEUE_SIZE environment variable.

`connector_max_idle_time`

Data type: Integer

Sets STORM_WEBDAV_CONNECTOR_MAX_IDLE_TIME environment variable.

`vo_map_files_enable`

Data type: Boolean

Sets STORM_WEBDAV_VO_MAP_FILES_ENABLE environment variable.

`vo_map_files_config_dir`

Data type: String

Sets STORM_WEBDAV_VO_MAP_FILES_CONFIG_DIR environment variable.

`vo_map_files_refresh_interval`

Data type: Integer

Sets STORM_WEBDAV_VO_MAP_FILES_REFRESH_INTERVAL environment variable.

`tpc_max_connections`

Data type: Integer

Sets STORM_WEBDAV_TPC_MAX_CONNECTIONS environment variable.

`tpc_max_connections_per_route`

Data type: Integer

Sets STORM_WEBDAV_TPC_MAX_CONNECTIONS_PER_ROUTE environment variable.

`tpc_verify_checksum`

Data type: Boolean

Sets STORM_WEBDAV_TPC_VERIFY_CHECKSUM environment variable.

`tpc_timeout_in_secs`

Data type: Integer

Sets STORM_WEBDAV_TPC_TIMEOUT_IN_SECS environment variable.

`tpc_tls_protocol`

Data type: String

Sets STORM_WEBDAV_TPC_TLS_PROTOCOL environment variable.

`tpc_report_delay_secs`

Data type: Integer

Sets STORM_WEBDAV_TPC_REPORT_DELAY_SECS environment variable.

`tpc_enable_tls_client_auth`

Data type: Boolean

Sets STORM_WEBDAV_TPC_ENABLE_TLS_CLIENT_AUTH environment variable.

`tpc_progress_report_thread_pool_size`

Data type: Integer

Sets STORM_WEBDAV_TPC_PROGRESS_REPORT_THREAD_POOL_SIZE environment variable.

`tpc_enable_expect_continue_threshold`

Data type: Integer

Sets STORM_WEBDAV_TPC_ENABLE_EXPECT_CONTINUE_THRESHOLD environment variable.

`jvm_opts`

Data type: String

Sets part of STORM_WEBDAV_JVM_OPTS environment variable.

`authz_server_enable`

Data type: Boolean

Sets STORM_WEBDAV_AUTHZ_SERVER_ENABLE environment variable.

`authz_server_issuer`

Data type: String

Sets STORM_WEBDAV_AUTHZ_SERVER_ISSUER environment variable if authz_server_enable is true.

`authz_server_max_token_lifetime_sec`

Data type: Integer

Sets STORM_WEBDAV_AUTHZ_SERVER_MAX_TOKEN_LIFETIME_SEC environment variable if authz_server_enable is true.

`authz_server_secret`

Data type: String

Sets STORM_WEBDAV_AUTHZ_SERVER_SECRET environment variable if authz_server_enable is true.

`require_client_cert`

Data type: Boolean

Sets STORM_WEBDAV_REQUIRE_CLIENT_CERT environment variable.

`use_conscrypt`

Data type: Boolean

Sets STORM_WEBDAV_USE_CONSCRYPT environment variable.

`tpc_use_conscrypt`

Data type: Boolean

Sets STORM_WEBDAV_TPC_USE_CONSCRYPT environment variable.

`enable_http2`

Data type: Boolean

Sets STORM_WEBDAV_ENABLE_HTTP2 environment variable.

`debug`

Data type: Boolean

Sets part of STORM_WEBDAV_JVM_OPTS environment variable. It enables remote debug.

`debug_port`

Data type: Integer

Sets part of STORM_WEBDAV_JVM_OPTS environment variable. It sets the remote debug port if remote debug is enabled.

`debug_suspend`

Data type: Boolean

Sets part of STORM_WEBDAV_JVM_OPTS environment variable. It sets debug suspend value in case remote debug is enabled.

`storm_limit_nofile`

Data type: Integer

Sets LimitNOFILE value.

`user`

Data type: String

Sets Unix user which runs the service

`tape_wellknown_source`

Data type: String

Sets the absolute path of the source file containing the tape REST API well-known endpoints

`tls_protocol`

Data type: String

Sets TLS protocol for non-TPC requests

`checksum_filter_enabled`

Data type: Boolean

Enable checksum filter which adds checksum as an header following RFC 3230.

`macaroon_filter_enabled`

Data type: Boolean

Enable Macaroon filter to process Macaroon tokens. Requires authz server enabled.

`file_buffer_size`

Data type: Integer

Buffer size for both internal and third-party copy requests. Valid values are numbers greater/equals than 4096.

`voms_trust_store_dir`

Data type: String

VOMS Trust Store directory

`voms_trust_store_refresh_interval_sec`

Data type: Integer

VOMS Trust Store refresh interval

`voms_cache_enabled`

Data type: Boolean

Enable caching for VOMS certificate validation

`voms_cache_entry_lifetime_sec`

Data type: Integer

Cache entries lifetime, used if caching for VOMS certificate validation is enabled

`storm::webdav::config`

StoRM WebDAV config class

`storm::webdav::install`

StoRM WebDAV install class

`storm::webdav::service`

StoRM WebDAV service class

Defined types

`storm::backend::service_conf_file`

Starting from Puppet module v2.0.0, site administrators can inject one or more .conf files into /etc/systemd/system/storm-backend-server.service.d directory.

Examples

class { 'storm::backend':
  hostname => 'storm-backend.test.example',
  # ...
}
storm::backend::service_conf_file { 'override.conf':
  source => '/path/to/my/override.conf',
}

Parameters

The following parameters are available in the storm::backend::service_conf_file defined type:

source

`source`

Data type: String

The source of file resource. It can be an absolute path or a Puppet module relative path.

`storm::backend::storage_site_report`

by using this defined type.

Examples

class { 'storm::backend':
  hostname => 'storm-backend.test.example',
  # ...
}
storm::backend::storage_site_report { 'storage-site-report':
  report_path => '/storage/info/report.json', # the internal storage area path
  minute      => '*/20', # set cron's minute
}

Parameters

The following parameters are available in the storm::backend::storage_site_report defined type:

report_path
minute

`report_path`

Data type: String

The full path of the generated report. Usually it points to a "info" storage area.

`minute`

Data type: String

The cron job's minute parameter. Refer to Resource Type cron.

Default value: '*/30'

`storm::rootdir`

StoRM main storage area root directory defined resource

Parameters

The following parameters are available in the storm::rootdir defined type:

mode

`mode`

Data type: String

Default value: '0755'

`storm::sarootdir`

StoRM Storage Area root directory defined resource

Parameters

The following parameters are available in the storm::sarootdir defined type:

mode

`mode`

Data type: String

Default value: '0750'

`storm::webdav::application_file`

Starting from Puppet module v2.0.0, the management of application.yml file has been removed from storm::webdav class. Site administrators can edit their own configuration files or use this defined type to inject one or more YAML files into the proper directory.

Examples

class { 'storm::webdav':
  hostnames => ['storm-webdav.test.example', 'alias-for-storm-webdav.test.example'],
}
storm::webdav::application_file { 'application.yml':
  source => '/path/to/my/application.yml',
}
storm::webdav::application_file { 'application-wlcg.yml':
  source => '/path/to/my/application-wlcg.yml',
}

Parameters

The following parameters are available in the storm::webdav::application_file defined type:

source

`source`

Data type: String

The source of file resource. It can be an absolute path or a Puppet module relative path.

`storm::webdav::drop_in_file`

Use this define to inject one or more .conf files into /etc/systemd/system/storm-webdav.service.d directory.

Examples

class { 'storm::webdav':
  # storm webdav parameters
}
storm::webdav::drop_in_file { 'override.conf':
  source => '/path/to/my/override.conf',
}

Parameters

The following parameters are available in the storm::webdav::drop_in_file defined type:

source

`source`

Data type: String

The source of file resource. It can be an absolute path or a Puppet module relative path.

`storm::webdav::storage_area_file`

Storage Areas can be configured singularly by using this defined type. This strategy allows site administrators to keep their manifests unaware of the improvements on StoRM WebDAV code. For example, if a new property is added into Storage Area configuration files, you haven't to update your Puppet module and all the service configuration will continue working.

Examples

class { 'storm::webdav':
  hostnames => ['storm-webdav.test.example', 'alias-for-storm-webdav.test.example'],
}
storm::webdav::storage_area_file { 'test.vo.properties':
  source => '/path/to/my/test.vo.properties',
}
storm::webdav::storage_area_file { 'test.vo.2.properties':
  source => '/path/to/my/test.vo.2.properties',
}

Parameters

The following parameters are available in the storm::webdav::storage_area_file defined type:

source

`source`

Data type: String

The source of file resource. It can be an absolute path or a Puppet module relative path.

Data types

`Storm::Backend::Acl`

The ACL type for storm-backend-server

Alias of

Struct[{
  group => String,
  permission => Enum['R', 'W', 'RW'],
}]

`Storm::Backend::AclMode`

The AclMode type for storm-backend-server

Alias of

Enum['AoT', 'JiT']

`Storm::Backend::BalanceStrategy`

The BalanceStrategy type for storm-backend-server

Alias of

Enum['round-robin', 'smart-rr', 'random', 'weight']

`Storm::Backend::Endpoint`

The Endpoint type for storm-backend-server

Alias of

Struct[{
  schema   => Enum['rfio','xroot','gsiftp','srm','http','https'],
  hostname => String,
  port     => Integer,
  path     => String,
}]

`Storm::Backend::FileSystem`

The FileSystem type for storm-backend-server

Alias of

Struct[{
  type => Enum['ext3', 'gpfs'],
  driver => Enum['posixfs', 'gpfs', 'test'],
  space_system => Optional[Enum['MockSpaceSystem', 'GPFSSpaceSystem']],
}]

`Storm::Backend::FsDriver`

The FsDriver type for storm-backend-server

Alias of

Enum['posixfs', 'gpfs', 'test']

`Storm::Backend::FsType`

The FsType type for storm-backend-server

Alias of

Enum['posixfs', 'gpfs', 'test']

`Storm::Backend::Gsiftp`

The Gsiftp type for storm-backend-server

Alias of

Struct[{
  pool => Storm::Backend::Pool,
}]

`Storm::Backend::GsiftpPoolMember`

The GsiftpPoolMember type for storm-backend-server

Alias of

Struct[{
  hostname => String,
  port     => Optional[Integer],
  weight   => Optional[Integer],
}]

`Storm::Backend::Pool`

The Pool type for storm-backend-server

Alias of

Struct[{
  balance_strategy => Optional[Enum['round-robin', 'smart-rr', 'random', 'weight']],
  members          => Array[Struct[{
    hostname => String,
    port     => Optional[Integer],
    weigth   => Optional[Integer],
  }]],
}]

`Storm::Backend::Quota`

The Quota type for storm-backend-server

Alias of

Struct[{
  device => String,
  type   => Enum['username', 'group', 'fileset'],
  value  => String,
}]

`Storm::Backend::Rfio`

The Rfio type for storm-backend-server

Alias of

Struct[{
  hostname => String,
  port => Optional[Integer],
}]

`Storm::Backend::SrmPoolMember`

The SrmPoolMember type for storm-backend-server

Alias of

Struct[{
  hostname => String,
  port     => Optional[Integer],
}]

`Storm::Backend::StorageArea`

The storage area type for storm-backend-server

Alias of

Struct[{
    name                         => String,
    root_path                    => String,
    access_points                => Array[String],
    vos                          => Array[String],
    fs_type                      => Optional[Storm::Backend::FsType],
    space_token                  => Optional[String],
    authz                        => Optional[String],
    storage_class                => Optional[Storm::Backend::StorageClass],
    online_size                  => Integer,
    nearline_size                => Optional[Integer],
    acl_mode                     => Optional[Storm::Backend::AclMode],
    default_acl_list             => Optional[Array[Storm::Backend::Acl]],
    quota                        => Optional[Storm::Backend::Quota],
    dn_regex                     => Optional[String],
    anonymous_http_read          => Optional[Boolean],
    transfer_protocols           => Optional[Array[Storm::Backend::TransferProtocol]],
    rfio_hostname                => Optional[String],
    rfio_port                    => Optional[Integer],
    xroot_hostname               => Optional[String],
    xroot_port                   => Optional[Integer],
    gsiftp_pool_balance_strategy => Optional[Storm::Backend::BalanceStrategy],
    gsiftp_pool_members          => Optional[Array[Storm::Backend::GsiftpPoolMember]],
    webdav_pool_balance_strategy => Optional[Storm::Backend::BalanceStrategy],
    webdav_pool_members          => Optional[Array[Storm::Backend::WebdavPoolMember]],
}]

`Storm::Backend::StorageClass`

The StorageClass type for storm-backend-server

Alias of

Enum['T0D1', 'T1D0', 'T1D1']

`Storm::Backend::TransferProtocol`

The TransferProtocol type for storm-backend-server

Alias of

Enum['file', 'gsiftp', 'xroot', 'webdav']

`Storm::Backend::Webdav`

The WebDAV type for storm-backend-server

Alias of

Struct[{
  pool => Array[Struct[{
    hostname => String,
    port     => Optional[Integer],
  }]],
}]

`Storm::Backend::WebdavPoolMember`

The WebdavPoolMember type for storm-backend-server

Alias of

Struct[{
  hostname => String,
  http_port => Optional[Integer],
  https_port => Optional[Integer],
}]

`Storm::Backend::Xroot`

The Xroot type for storm-backend-server

Alias of

Struct[{
  hostname => String,
  port => Optional[Integer],
}]

`Storm::CustomRepo`

The storage area type for storm-webdav

Alias of

Struct[{
  name    => String,
  baseurl => String,
}]

`Storm::Webdav::StorageArea`

The storage area type for storm-webdav.

Alias of

Struct[{
  name                           => String,
  root_path                      => String,
  filesystem_type                => Optional[String],
  access_points                  => Optional[Array[String]],
  vos                            => Optional[Array[String]],
  orgs                           => Optional[Array[String]],
  authenticated_read_enabled     => Optional[Boolean],
  anonymous_read_enabled         => Optional[Boolean],
  vo_map_enabled                 => Optional[Boolean],
  vo_map_grants_write_permission => Optional[Boolean],
  orgs_grant_read_permission     => Optional[Boolean],
  orgs_grant_write_permission    => Optional[Boolean],
  wlcg_scope_authz_enabled       => Optional[Boolean],
  fine_grained_authz_enabled     => Optional[Boolean],
}]

Changelog

All notable changes to this project will be documented in this file.

[4.1.0]

Added the opportunity to specify an external source for Backend's storm.properties file. Use storm::backend::manage_storm_properties and storm::backend::path_storm_properties.

[4.0.1]

Added new StoRM WebDAV parameters for the related environment variables:
- storm::webdav::tls_protocol
- storm::webdav::checksum_filter_enabled
- storm::webdav::macaroon_filter_enabled
- storm::webdav::file_buffer_size
- storm::webdav::voms_trust_store_dir
- storm::webdav::voms_trust_store_refresh_interval_sec
- storm::webdav::voms_cache_enabled
- storm::webdav::voms_cache_entry_lifetime_sec

[4.0.0]

Puppet version >= 7.0.0 is required
Added storm::gridftp::data_interface parameter in order to support a configuration behind a public IP
Added storm::gridftp::log_level parameter in order to allow logging level
Removed storm::webdav::ensure_empty_storage_area_dir: storage area directory is now automatically purged of not managed properties files
Added storm::webdav::tpc_enable_expect_continue_threshold in order to set a threshold from which an header with Expect: 100 continue is added
Added storm::webdav::user parameter in order to allow the setting of a custom user used to run the service
Added storm::webdav::tape_wellknown_source parameter in order to specify which JSON file has to be loaded as content of the well-known endpoint for WLCG Tape REST API implementation
Within update-site-report script, JSON file is now moved and not copied to avoid the growth of useless files in tmp directory
Removed storm::backend::debug* parameters
Removed storm::backend::install_native_libs_gpfs parameter because the installation of the GPFS libs is done if at least one storage area has fs_type GPFS.
Added a retry mechanism if StoRM Info Provider configure command fails
Removed the custom management of systemctl daemon reload because now it's supported by Puppet 7

[3.4.0]

Removed useless storm::frontend parameters gridmap_dir and gridmap_file

[3.3.2]

Changed default value to an empty array for vos field of Storm::Webdav::StorageArea
Added pool of webdav endpoints into backend's namespace.xml

[3.3.1]

Added missing Argus's resource-id parameter within Frontend's configuration.
Added support for using an external file for StoRM Frontend's configuration file.
Removed storm::frontend::security_enable_mapping parameter.
The storm::frontend::be_xmlrpc_host value is initialized with local FQDN.

[3.3.0]

Added missing StoRM WebDAV configuration parameters
StoRM WebDAV hostname list initialized with local FQDN
Added storm::webdav::drop_in_file define in order to allow adding custom drop-in files to be stored in /etc/systemd/system/storm-webdav.service.d
Added storm::webdav::tpc_max_connections_per_route, storm::webdav::tpc_timeout_in_secs, storm::webdav::tpc_tls_protocol, storm::webdav::tpc_report_delay_secs, storm::webdav::tpc_enable_tls_client_auth, storm::webdav::tpc_progress_report_thread_pool_size parameters.
Removed storm::webdav::manage_storage_areas parameter. The logic has been simplified to managing storage areas only if storm::webdav::storage_areas list is defined.
Added storm::webdav::ensure_empty_storage_area_dir parameter to ensure a cleaned storage area directory before adding the .properties files.
Increased StoRM WebDAV default heap size to 1024m and default timeout for TPC to 30 seconds

[3.2.1]

Fixed permissions on info-provider configuration file

[3.2.0]

Added daemon reload on backend and webdav restart command

[3.1.1]

Improved README. No changes on module.

[3.1.0]

Added mode parameter to storm::rootdir and storm::sarootdir defined types
Increased default value for wait_timeout in storm::db::override_options

[3.0.0]

Converted to pdk project
Removed storm::storage class
Added storm::sarootdir defined type
Use hiera YAML files for all default parameters
Removed all params.pp files

[2.2.1] - 2020-11-09

Fixed type into data file

[2.2.0] - 2020-11-09

Moved default parameters into YAML data files
Deleted params.pp files
Added hiera.yaml

[2.1.0] - 2020-10-29

Moved gridftp.conf into /etc/grid-security directory

[2.0.1] - 2020-10-19

Fixed CHANGELOG
Improved documentation.

[2.0.0] - 2020-10-19

Added defined types for application-*.yml and storage area files used by WebDAV class.
Removed direct support for application.yml template file.
Added defined types for service conf files of Backend.
Added defined type for storage site reporting.
Added tests.
Removed docker deploy image.

[1.1.1] - 2020-10-06

Removed log directory management

[1.1.0] - 2020-10-05

Removed storm::mapping class

[1.0.5] - 2020-10-02

Added path-authz.db support to StoRM Backend

[1.0.0] - 2020-09-25

Moved database creation and MySQL/MariaDB installation on a separated class storm::db
Backend class only import SQL script of databases, only if database or table version not exists
WebDAV class by default doesn't manage application.yml file. It can be optionally managed from a source or from variables.
WebDAV class by default manage storage areas. A source directory or the array of storage areas can be provided.
Mapping class also manages VOMS roles and multiple groups.
Removed epel and umd repo from repo class
Removed testca and testvos classes

[0.2.3] - 2020-05-25

Changed

Allow enable/disable StoRM DSI module for gridftp

[0.2.1] - 2020-05-25

Changed

Updated README.md

[0.2.0] - 2020-05-15

Added

Support for StoRM Frontend service configuration

Changed

Fixed ignored WebDAV HTTP2 and Conscrypt options
Allow redirection of LCMAPS logging to a particular file
Fixed conflict on storm user/group creation when the same node install WebDAV and Frontend services

[0.1.0] - 2019-10-09

Added

Support for StoRM WebDAV service configuration
Support for StoRM GridFTP server configuration
Documentation at REFERENCE.md
Documentation at https://italiangrid.github.io/storm-puppet-module

Modules

Discover

Contribute

Puppet

Premium features

Downloads

Resources

About Forge

Getting Started

storm

Version information

This version is compatible with:

Start using this module

Documentation

StoRM puppet module

Description

Setup

Usage

StoRM Backend class

Enable HTTP as transfer protocol for SRM

StoRM Frontend class

StoRM WebDAV class

StoRM GridFTP class

StoRM database class

StoRM repo class

StoRM users class

Documentation

Developers

Limitations

Reference

Table of Contents

Classes

Defined types

Data types

Classes

storm::backend

Examples

Example of usage

Parameters

hostname

db_hostname

db_username

db_password

xroot_hostname

xroot_port

gsiftp_pool_balance_strategy

gsiftp_pool_members

webdav_pool_balance_strategy

webdav_pool_members

srm_pool_members

transfer_protocols

fs_type

storage_areas

frontend_public_host

frontend_port

directory_automatic_creation

directory_writeperm

rest_services_port

rest_services_max_threads

rest_services_max_queue_size

xmlrpc_unsecure_server_port

xmlrpc_maxthread

xmlrpc_max_queue_size

xmlrpc_security_enabled

xmlrpc_security_token

ptg_skip_acl_setup

pinlifetime_default

pinlifetime_maximum

sanity_check_enabled

service_du_enabled

service_du_delay

service_du_interval

max_ls_entries

gc_pinnedfiles_cleaning_delay

gc_pinnedfiles_cleaning_interval

gc_purge_enabled

gc_purge_interval

gc_purge_size

gc_expired_request_time

gc_expired_inprogress_time

`storm::backend`

`hostname`

`db_hostname`

`db_username`

`db_password`

`xroot_hostname`

`xroot_port`

`gsiftp_pool_balance_strategy`

`gsiftp_pool_members`

`webdav_pool_balance_strategy`

`webdav_pool_members`

`srm_pool_members`

`transfer_protocols`

`fs_type`

`storage_areas`

`frontend_public_host`

`frontend_port`

`directory_automatic_creation`

`directory_writeperm`

`rest_services_port`

`rest_services_max_threads`

`rest_services_max_queue_size`

`xmlrpc_unsecure_server_port`

`xmlrpc_maxthread`

`xmlrpc_max_queue_size`

`xmlrpc_security_enabled`

`xmlrpc_security_token`

`ptg_skip_acl_setup`

`pinlifetime_default`

`pinlifetime_maximum`

`sanity_check_enabled`

`service_du_enabled`

`service_du_delay`

`service_du_interval`

`max_ls_entries`

`gc_pinnedfiles_cleaning_delay`

`gc_pinnedfiles_cleaning_interval`

`gc_purge_enabled`

`gc_purge_interval`

`gc_purge_size`

`gc_expired_request_time`

`gc_expired_inprogress_time`

`gc_ptp_transit_interval`

`gc_ptp_transit_start_delay`

`extraslashes_file`

`extraslashes_root`

`extraslashes_gsiftp`

`db_connection_pool_enabled`

`db_connection_pool_max_active`

`db_connection_pool_max_wait`

`asynch_db_reconnect_period`

`asynch_db_delay_period`

`asynch_picking_initial_delay`

`asynch_picking_time_interval`

`asynch_picking_max_batch_size`

`requests_scheduler_core_size`

`requests_scheduler_max_size`

`requests_scheduler_queue_size`

`ptp_requests_scheduler_core_size`

`ptp_requests_scheduler_max_size`

`ptp_requests_scheduler_queue_size`

`ptg_requests_scheduler_core_size`

`ptg_requests_scheduler_max_size`

`ptg_requests_scheduler_queue_size`

`bol_requests_scheduler_core_size`

`bol_requests_scheduler_max_size`

`bol_requests_scheduler_queue_size`

`info_config_file`

`info_sitename`

`info_storage_default_root`

`info_endpoint_quality_level`

`info_webdav_pool_list`

`info_frontend_host_list`

`jvm_options`

`jmx`

`jmx_options`

`lcmaps_db_file`

`lcmaps_policy_name`

`lcmaps_log_file`

`lcmaps_debug_level`