StoRM puppet module

Table of Contents:

• Description
• Setup
• Usage
  • StoRM Backend class
  • StoRM Frontend class
  • StoRM WebDAV class
  • StoRM GridFTP class
  • StoRM database class
  • StoRM repo class
  • StoRM users class
• Limitations - OS compatibility

Description

StoRM Puppet module allows administrators to configure StoRM services deployed on CentOS 7 and AlmaLinux/RedHat 9.

The supported services are:

• StoRM Backend
• StoRM Frontend
• StoRM WebDAV
• StoRM Globus GridFTP

Setup

StoRM Puppet module is available on puppet forge:

puppet module install cnafsd-storm

You can also build and install module from source code as follow:

git clone https://github.com/italiangrid/storm-puppet-module.git
cd storm-puppet-module
pdk build
puppet module install ./pkg/cnafsd-storm-*.tar.gz

Usage

This Puppet module allows site administrators to properly configure StoRM services. This module provides some classes related to the main components and also some utility classes those can be used to configure StoRM repositories, StoRM users, VO pool accounts, LCMAPS and storage directories.

Component classes:

• StoRM Backend class
• StoRM Frontend class
• StoRM WebDAV class
• StoRM GridFTP class

Utility classes:

• StoRM database class
• StoRM repo class
• StoRM users class

StoRM Backend class

Prerequisites: A MySQL or MariaDB server with StoRM databases must exist. Databases can be empty. If you want to use this module to install MySQL client and server and init databases, please read about StoRM database utility class.

The Backend class installs storm-backend-server and all its related packages, such as storm-native-libs, storm-native-libs-gpfs in case GPFS is used as filesystem and storm-dynamic-info-provider.

The Backend class configures storm-backend-server service by managing the following files:

• /etc/storm/backend-server/storm.properties
• /etc/storm/backend-server/namespace.xml
• /etc/systemd/system/storm-backend-server.service.d/storm-backend-server.conf
• /etc/systemd/system/storm-backend-server.service.d/filelimit.conf

and deploys StoRM databases. In addiction, this class configures and run storm-info-provider by managing the following file:

• /etc/storm/info-provider/storm-yaim-variables.conf.

The whole list of StoRM Backend class parameters can be found here.

Example of StoRM Backend configuration:

class { 'storm::backend':
  db_password           => 'secret-password',
  transfer_protocols    => ['file', 'gsiftp', 'webdav'],
  xmlrpc_security_token => 'secret-token',
  srm_pool_members      => [
    {
      'hostname' => 'frontend-public-host.example',
    },
    {
      'hostname' => 'other-frontend-host.example',
    }
  ],
  gsiftp_pool_members   => [
    {
      'hostname' => 'gridftp-01.example',
    },
    {
      'hostname' => 'gridftp-02.example',
    }
  ],
  webdav_pool_members   => [
    {
      'hostname' => webdav-01.example,
    },
    {
      'hostname' => webdav-02.example,
    }
  ],
  storage_areas         => [
    {
      'name'          => 'dteam-disk',
      'root_path'     => '/storage/disk',
      'access_points' => ['/disk'],
      'vos'           => ['dteam'],
      'online_size'   => 40,
    },
    {
      'name'          => 'dteam-tape',
      'root_path'     => '/storage/tape',
      'access_points' => ['/tape'],
      'vos'           => ['dteam'],
      'online_size'   => 40,
      'nearline_size' => 80,
      'fs_type'       => 'gpfs',
      'storage_class' => 'T1D0',
    },
  ],
}

Starting from Puppet module v2.0.0, the management of Storage Site Report has been improved. Site administrators can add script and cron described in the how-to using a defined type storm::backend::storage_site_report. For example:

storm::backend::storage_site_report { 'storage-site-report':
  report_path => '/storage/info/report.json', # the internal storage area path
  minute      => '*/20', # set cron's minute
}

Enable HTTP as transfer protocol for SRM

To enable HTTP as transfer protocol for SRM prepare-to-get and prepare-to-put requests, you must add webdav protocol to the list of your transfer_protocols and define at least one member for webdav_pool_members. You can re-define the default list of transfer protocols by adding your storm::backend::transfer_protocols variable and/or you can override this list by adding a specific transfer_protocols for each storage area:

class { 'storm::backend':
  # ...
  'webdav_pool_members' => [
    {
      'hostname' => webdav.test.example,
    },
  ],
  # defines the default list of transfer protocols for each storage area:
  'transfer_protocols'  => ['file', 'gsiftp', 'webdav'], 
  'storage_areas'       => [
    {
      'name'          => 'sa-http-enabled',
      'root_path'     => '/storage/sa-http-enabled',
      'access_points' => ['/sa-http-enabled'],
      'vos'           => ['test.vo'],
      'online_size'   => 40,
    },
    {
      'name'               => 'sa-no-http-enabled',
      'root_path'          => '/storage/sa-no-http-enabled',
      'access_points'      => ['/sa-no-http-enabled'],
      'vos'                => ['test.vo'],
      'online_size'        => 40,
      # disable webdav protocol for this storage area
      'transfer_protocols' => ['file', 'gsiftp'],
    },
    # ...
  ],
  # ...
}

The manifest.pp showed above includes the HTTP transfer protocol for all the storage area defined. By default, storm::backend::transfer_protocols includes only file and gsiftp.

StoRM Frontend class

The StoRM Frontend class installs storm-frontend-mp and all the releated packages and configures storm-frontend-server service by managing the following files:

• /etc/storm/frontend-server/storm-frontend-server.conf
• /etc/sysconfig/storm-frontend-server

The whole list of StoRM Frontend class parameters can be found here.

Example of StoRM Frontend configuration:

class { 'storm::frontend':
  be_xmlrpc_host  => 'backend.test.example',
  be_xmlrpc_token => 'NS4kYAZuR65XJCq',
  db_host         => 'backend.test.example',
  db_user         => 'storm',
  db_passwd       => 'secret-password',
}

StoRM WebDAV class

The StoRM WebDAV class installs storm-webdav rpm and configures storm-webdav service by managing the following files:

• the systemd override files filelimit.conf and storm-webdav.conf stored into /etc/systemd/system/storm-webdav.service.d;
• the storage areas property files stored into /etc/storm/webdav/sa.d (optional);

The whole list of StoRM Webdav class parameters can be found here.

Example of StoRM WebDAV configuration:

class { 'storm::webdav':
  storage_areas => [
    {
      name                       => 'test.vo',
      root_path                  => '/storage/test.vo',
      access_points              => ['/test.vo'],
      vos                        => ['test.vo'],
    },
    {
      name                       => 'test.vo.2',
      root_path                  => '/storage/test.vo.2',
      access_points              => ['/test.vo.2', '/alias'],
      vos                        => ['test.vo.2'],
      authenticated_read_enabled => true,
    },
  ],
  hostnames => ['storm-webdav.test.example', 'alias-for-storm-webdav.test.example'],
}

Storage Areas can also be configured singularly by using the defined type storm::webdav::storage_area_file. This strategy allows site administrators to keep their manifests unaware of the improvements on StoRM WebDAV code. For example, if a new property is added into Storage Area configuration files, you haven't to update your Puppet module and all the service configuration will continue working.

Example of Storage Areas configuration done with storm::webdav::storage_area_file:

class { 'storm::webdav':
  hostnames => ['storm-webdav.test.example', 'alias-for-storm-webdav.test.example'],
}

storm::webdav::storage_area_file { 'test.vo.properties':
  source => '/path/to/my/test.vo.properties',
}

storm::webdav::storage_area_file { 'test.vo.2.properties':
  source => '/path/to/my/test.vo.2.properties',
}

Starting from Puppet module v2.0.0, the management of application.yml file has been removed from storm::webdav class. Site administrators can edit their own configuration files or use a defined type storm::webdav::application_file to inject also one or more YAML files into the proper directory. For example:

class { 'storm::webdav':
  jvm_opts => '-Xms1024m -Xmx1024m -Dspring.profiles.active=extra',
}

storm::webdav::application_file { 'application.yml':
  source => '/path/to/my/application.yml',
}

storm::webdav::application_file { 'application-extra.yml':
  source => '/path/to/my/application-extra.yml',
}

StoRM GridFTP class

The StoRM GridFTP class installs storm-globus-gridftp-mp and configures storm-globus-gridftp service by managing the following files:

• /etc/grid-security/gridftp.conf, the main configuration file;
• /etc/sysconfig/storm-globus-gridftp, with the environment variables.

The whole list of StoRM GridFTP class parameters can be found here.

Examples of StoRM Gridftp configuration:

class { 'storm::gridftp':
  redirect_lcmaps_log => true,
  llgt_log_file       => '/var/log/storm/storm-gridftp-lcmaps.log',
}

StoRM database class

The StoRM database utility class installs mariadb server and releated rpms and configures mysql service by managing the following files:

• /etc/my.cnf.d/server.cnf;
• /etc/systemd/system/mariadb.service.d/limits.conf.

The whole list of StoRM Database class parameters can be found here.

Examples of StoRM Database usage:

class { 'storm::db':
  root_password => 'supersupersecretword',
  storm_password => 'supersecretword',
}

StoRM repo class

The StoRM repo utility class creates all the StoRM YUM repositories: stable, beta, nightly. By default, only stable repo is enabled. You can also add extra repositories to install.

The whole list of StoRM repo class parameters can be found here.

Examples of StoRM Repo usage:

class { 'storm::repo':
  enabled => ['stable', 'beta'],
}

StoRM users class

The StoRM users utility class creates the default StoRM users and groups.

Use:

include storm::users

to create default scenario:

• storm group with id 1100
• edguser group with id 1101
• storm user with id 1100, member of storm and edguser groups
• edguser user with id 1101, member of edguser and storm groups

You can also customize and create your own users and groups as follow:

class { 'storm::users':
  groups => {
    infosys => {
      gid => '996',
    },
  },
  users  => {
    edguser => {
      comment => 'Edguser user',
      groups  => [ edguser, infosys, storm, ],
      uid     => '995',
      gid     => '995',
      home    => '/home/edguser',
    },
    storm   => {
      comment => 'StoRM user',
      groups  => [ storm, edguser, ],
      uid     => '991',
      gid     => '991',
      home    => '/home/storm',
    },
  },
}

The whole list of StoRM repo class parameters can be found here.

Documentation

You can find all the info about module classes and parameters at:

• StoRM Puppet module main site doc
• REFERENCE.md

Developers

Run tests with:

pdk test unit

or

bundle update
bundle install
bundle exec rake spec

Validate code with:

pdk validate

Limitations

It works only on RedHat CentOS 7 distribution or AlmaLinux/RedHat 9.

Reference

Table of Contents

Classes

• storm::backend: StoRM Backend puppet module
• storm::backend::config: StoRM Backend config class
• storm::backend::configdb: StoRM Backend DB config class
• storm::backend::install: StoRM Backend install class
• storm::backend::service: StoRM Backend service class
• storm::db: Install MariaDB server and client, create empty databases 'storm_db' and 'storm_be_ISAM', add storm user and all the necessary grants.
• storm::frontend: StoRM Frontend puppet module
• storm::frontend::config: StoRM Frontend config class
• storm::frontend::install: StoRM Frontend install class
• storm::frontend::service: StoRM Frontend service class
• storm::gridftp: StoRM GridFTP puppet module
• storm::gridftp::config: StoRM GridFTP config class
• storm::gridftp::install: StoRM GridFTP install class
• storm::gridftp::service: StoRM GridFTP service class
• storm::repo: Choose which StoRM repository you want to intall and enable. Also a custom list of repository URL can be specified.
• storm::users: StoRM accounts configuration
• storm::webdav: StoRM WebDAV puppet module
• storm::webdav::config: StoRM WebDAV config class
• storm::webdav::install: StoRM WebDAV install class
• storm::webdav::service: StoRM WebDAV service class

Defined types

• storm::backend::service_conf_file: Starting from Puppet module v2.0.0, site administrators can inject one or more .conf files into /etc/systemd/system/storm-backend-server.service.d directory.
• storm::backend::storage_site_report: Starting from Puppet module v2.0.0, the management of Storage Site Report has been improved. Site administrators can add script and cron described in the how-to
• storm::rootdir: StoRM main storage area root directory defined resource
• storm::sarootdir: StoRM Storage Area root directory defined resource
• storm::webdav::application_file: Starting from Puppet module v2.0.0, the management of application.yml file has been removed from storm::webdav class. Site administrators can edit their own configuration files or use this defined type to inject one or more YAML files into the proper directory.
• storm::webdav::drop_in_file: Use this define to inject one or more .conf files into /etc/systemd/system/storm-webdav.service.d directory.
• storm::webdav::storage_area_file: Storage Areas can be configured singularly by using this defined type. This strategy allows site administrators to keep their manifests unaware of the improvements on StoRM WebDAV code. For example, if a new property is added into Storage Area configuration files, you haven't to update your Puppet module and all the service configuration will continue working.

Data types

• Storm::Backend::Acl: The ACL type for storm-backend-server
• Storm::Backend::AclMode: The AclMode type for storm-backend-server
• Storm::Backend::BalanceStrategy: The BalanceStrategy type for storm-backend-server
• Storm::Backend::Endpoint: The Endpoint type for storm-backend-server
• Storm::Backend::FileSystem: The FileSystem type for storm-backend-server
• Storm::Backend::FsDriver: The FsDriver type for storm-backend-server
• Storm::Backend::FsType: The FsType type for storm-backend-server
• Storm::Backend::Gsiftp: The Gsiftp type for storm-backend-server
• Storm::Backend::GsiftpPoolMember: The GsiftpPoolMember type for storm-backend-server
• Storm::Backend::Pool: The Pool type for storm-backend-server
• Storm::Backend::Quota: The Quota type for storm-backend-server
• Storm::Backend::Rfio: The Rfio type for storm-backend-server
• Storm::Backend::SrmPoolMember: The SrmPoolMember type for storm-backend-server
• Storm::Backend::StorageArea: The storage area type for storm-backend-server
• Storm::Backend::StorageClass: The StorageClass type for storm-backend-server
• Storm::Backend::TransferProtocol: The TransferProtocol type for storm-backend-server
• Storm::Backend::Webdav: The WebDAV type for storm-backend-server
• Storm::Backend::WebdavPoolMember: The WebdavPoolMember type for storm-backend-server
• Storm::Backend::Xroot: The Xroot type for storm-backend-server
• Storm::CustomRepo: The storage area type for storm-webdav
• Storm::Webdav::StorageArea: The storage area type for storm-webdav.

Classes

storm::backend

StoRM Backend puppet module

Examples

Example of usage

class { 'storm::backend':
  hostname => 'backend.test.example',
  db_username => 'storm',
  db_password => 'bluemoon',
  srm_pool_members => [
    {
      'hostname' => 'frontend.test.example',
    }
  ],
  gsiftp_pool_members => [
    {
      'hostname' => 'gridftp.test.example',
    },
  ],
  webdav_pool_members => [
    {
      'hostname' => 'webdav.test.example',
    },
  ],
  storage_areas => [
    {
      'name'               => 'test.vo',
      'root_path'          => '/storage/test.vo',
      'access_points'      => ['/test.vo'],
      'vos'                => ['test.vo'],
      'storage_class'      => 'T0D1',
      'online_size'        => 4,
      'transfer_protocols' => ['file', 'gsiftp', 'https'],
    },
  ],
}

Parameters

The following parameters are available in the storm::backend class:

• hostname
• db_hostname
• db_username
• db_password
• xroot_hostname
• xroot_port
• gsiftp_pool_balance_strategy
• gsiftp_pool_members
• webdav_pool_balance_strategy
• webdav_pool_members
• srm_pool_members
• transfer_protocols
• fs_type
• storage_areas
• frontend_public_host
• frontend_port
• directory_automatic_creation
• directory_writeperm
• rest_services_port
• rest_services_max_threads
• rest_services_max_queue_size
• xmlrpc_unsecure_server_port
• xmlrpc_maxthread
• xmlrpc_max_queue_size
• xmlrpc_security_enabled
• xmlrpc_security_token
• ptg_skip_acl_setup
• pinlifetime_default
• pinlifetime_maximum
• sanity_check_enabled
• service_du_enabled
• service_du_delay
• service_du_interval
• max_ls_entries
• gc_pinnedfiles_cleaning_delay
• gc_pinnedfiles_cleaning_interval
• gc_purge_enabled
• gc_purge_interval
• gc_purge_size
• gc_expired_request_time
• gc_expired_inprogress_time
• gc_ptp_transit_interval
• gc_ptp_transit_start_delay
• extraslashes_file
• extraslashes_root
• extraslashes_gsiftp
• db_connection_pool_enabled
• db_connection_pool_max_active
• db_connection_pool_max_wait
• asynch_db_reconnect_period
• asynch_db_delay_period
• asynch_picking_initial_delay
• asynch_picking_time_interval
• asynch_picking_max_batch_size
• requests_scheduler_core_size
• requests_scheduler_max_size
• requests_scheduler_queue_size
• ptp_requests_scheduler_core_size
• ptp_requests_scheduler_max_size
• ptp_requests_scheduler_queue_size
• ptg_requests_scheduler_core_size
• ptg_requests_scheduler_max_size
• ptg_requests_scheduler_queue_size
• bol_requests_scheduler_core_size
• bol_requests_scheduler_max_size
• bol_requests_scheduler_queue_size
• info_config_file
• info_sitename
• info_storage_default_root
• info_endpoint_quality_level
• info_webdav_pool_list
• info_frontend_host_list
• jvm_options
• jmx
• jmx_options
• lcmaps_db_file
• lcmaps_policy_name
• lcmaps_log_file
• lcmaps_debug_level
• http_turl_prefix
• storm_limit_nofile
• manage_path_authz_db
• path_authz_db_file
• manage_storm_properties
• path_storm_properties

hostname

Data type: String

StoRM Backend Fully Qualified Domain Name

Default value: $fqdn

db_hostname

Data type: String

Fully Qualified Domain Name of database hostname. Default value: hostname.

Default value: $hostname

db_username

Data type: String

The name of user used to connect to local database. Default: storm

db_password

Data type: String

Password for the user in db_storm_username. Default: bluemoon

xroot_hostname

Data type: String

Root server (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

Default value: $hostname

xroot_port

Data type: Integer

Root server port (default value for all Storage Areas).

gsiftp_pool_balance_strategy

Data type: Storm::Backend::BalanceStrategy

Load balancing strategy for GridFTP server pool (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration. Available values: round-robin, smart-rr, random, weight. Default value: round-robin. See Storm::Backend::BalanceStrategy.

gsiftp_pool_members

Data type: Array[Storm::Backend::GsiftpPoolMember]

Array of Storm::Backend::GsiftpPoolMember. GridFTP servers pool list (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

webdav_pool_balance_strategy

Data type: Storm::Backend::BalanceStrategy

Load balancing strategy for WebDAV server pools (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration. Available values: round-robin, smart-rr, random, weight. Default value: round-robin. See Storm::Backend::BalanceStrategy.

webdav_pool_members

Data type: Array[Storm::Backend::WebdavPoolMember]

Array of Storm::Backend::WebdavPoolMember. WebDAV endpoints pool list (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

srm_pool_members

Data type: Array[Storm::Backend::SrmPoolMember]

Array of Storm::Backend::SrmPoolMember. Frontend endpoints pool list (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

transfer_protocols

Data type: Array[Storm::Backend::TransferProtocol]

List of supported (and published) transfer protocols (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

fs_type

Data type: Storm::Backend::FsType

File System Type (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration. Available values: posixfs, gpfs and test. Default value: posixfs See Storm::Backend::FsType.

storage_areas

Data type: Array[Storm::Backend::StorageArea]

List of supported Storage Areas. Array of Storm::Backend::StorageArea.

frontend_public_host

Data type: String

StoRM Frontend service public host. It’s used by StoRM Info Provider to publish the SRM endpoint into the Resource BDII. Default value: hostname

Default value: $hostname

frontend_port

Data type: Integer

StoRM Frontend service port. Default value: 8444

directory_automatic_creation

Data type: Boolean

Flag to enable automatic missing directory creation upon srmPrepareToPut requests. Default: false.

directory_writeperm

Data type: Boolean

Flag to enable directory write permission setting upon srmMkDir requests on created directories. Default: false.

rest_services_port

Data type: Integer

REST services port. Default value: 9998

rest_services_max_threads

Data type: Integer

REST services max active requests. Default: 100

rest_services_max_queue_size

Data type: Integer

REST services max queue size of accepted requests. Default: 1000

xmlrpc_unsecure_server_port

Data type: Integer

Port to listen on for incoming XML-RPC connections from Frontends(s). Default: 8080

xmlrpc_maxthread

Data type: Integer

Number of threads managing XML-RPC connection from Frontends(s). A well sized value for this parameter have to be at least equal to the sum of the number of working threads in all Frontends. Default: 100.

xmlrpc_max_queue_size

Data type: Integer

Max number of accepted and queued XML-RPC connection from Frontends(s). Default: 1000

xmlrpc_security_enabled

Data type: Boolean

Whether the backend will require a token to be present for accepting XML-RPC requests. Default: true.

xmlrpc_security_token

Data type: String

The token that the backend will require to be present for accepting XML-RPC requests. Mandatory if xmlrpc_security_enabled is true.

ptg_skip_acl_setup

Data type: Boolean

Skip ACL setup for PtG requests. Default: false.

pinlifetime_default

Data type: Integer

Default PinLifetime in seconds used for pinning files in case of srmPrepareToPut or srmPrepareToGet operation without any pinLifetime specified. Default: 259200.

pinlifetime_maximum

Data type: Integer

Maximum PinLifetime allowed in seconds. Default: 1814400.

sanity_check_enabled

Data type: Boolean

Enable/disable sanity checks during bootstrap phase. Default: true.

service_du_enabled

Data type: Boolean

Flag to enable disk usage service. Default: false.

service_du_delay

Data type: Integer

The initial delay before the service is started (seconds). Default: 60.

service_du_interval

Data type: Integer

The interval in seconds between successive run. Default: 360.

max_ls_entries

Data type: Integer

Maximum number of entries returned by an srmLs call. Since in case of recursive srmLs results can be in order of million, this prevent a server overload. Default: 500.

gc_pinnedfiles_cleaning_delay

Data type: Integer

Initial delay before starting the reserved space, JIT ACLs and pinned files garbage collection process, in seconds. Default: 10.

gc_pinnedfiles_cleaning_interval

Data type: Integer

Time interval in seconds between successive purging run. Default: 300.

gc_purge_enabled

Data type: Boolean

Enable the request garbage collector. Default: true.

gc_purge_interval

Data type: Integer

Time interval in seconds between successive purging run. Default: 600.

gc_purge_size

Data type: Integer

Number of requests picked up for cleaning from the requests garbage collector at each run. This value is use also by Tape Recall Garbage Collector. Default: 800

gc_expired_request_time

Data type: Integer

Time in seconds to consider a request expired after its submission. Default: 604800 seconds (1 week). From StoRM 1.11.13 it is used also to identify how much time is needed to consider a completed recall task as cleanable.

gc_expired_inprogress_time

Data type: Integer

Time in seconds to consider an in-progress ptp request as expired. Default: 2592000 seconds (1 month).

gc_ptp_transit_interval

Data type: Integer

Time interval in seconds between successive expired put requests agent run. Default: 3000.

gc_ptp_transit_start_delay

Data type: Integer

Initial delay before starting the expired put requests agent process, in seconds. Default: 60

extraslashes_file

Data type: String

Add extra slashes after the “authority” part of a TURL for file protocol. Defaul: ''

extraslashes_root

Data type: String

Add extra slashes after the “authority” part of a TURL for xroot protocol. Default: '/'

extraslashes_gsiftp

Data type: String

Add extra slashes after the “authority” part of a TURL for gsiftp protocol. Default: '/'

db_connection_pool_enabled

Data type: Boolean

Enable the database connection pool. Default: true

db_connection_pool_max_active

Data type: Integer

Database connection pool max active connections. Default: 10

db_connection_pool_max_wait

Data type: Integer

Database connection pool max wait time to provide a connection. Default: 50

asynch_db_reconnect_period

Data type: Integer

Database connection refresh time intervall in seconds. Default: 18000

asynch_db_delay_period

Data type: Integer

Database connection refresh initial delay in seconds. Default: 30.

asynch_picking_initial_delay

Data type: Integer

Initial delay before starting to pick requests from the DB, in seconds. Default: 1.

asynch_picking_time_interval

Data type: Integer

Polling interval in seconds to pick up new SRM requests. Default: 2.

asynch_picking_max_batch_size

Data type: Integer

Maximum number of requests picked up at each polling time. Default: 100.

requests_scheduler_core_size

Data type: Integer

Crusher Scheduler worker pool base size. Default: 50.

requests_scheduler_max_size

Data type: Integer

Crusher Schedule worker pool max size. Default: 200.

requests_scheduler_queue_size

Data type: Integer

Request queue maximum size. Default: 2000.

ptp_requests_scheduler_core_size

Data type: Integer

PrepareToPut worker pool base size. Default: 50.

ptp_requests_scheduler_max_size

Data type: Integer

PrepareToPut worker pool max size. Default: 200.

ptp_requests_scheduler_queue_size

Data type: Integer

PrepareToPut request queue maximum size. Default: 1000.

ptg_requests_scheduler_core_size

Data type: Integer

PrepareToGet worker pool base size. Default: 50.

ptg_requests_scheduler_max_size

Data type: Integer

PrepareToGet worker pool max size. Default: 200.

ptg_requests_scheduler_queue_size

Data type: Integer

PrepareToGet request queue maximum size. Default: 2000.

bol_requests_scheduler_core_size

Data type: Integer

BringOnline worker pool base size. Default: 50.

bol_requests_scheduler_max_size

Data type: Integer

BringOnline Worker pool max size. Default: 200.

bol_requests_scheduler_queue_size

Data type: Integer

BringOnline request queue maximum size. Default: 2000.

info_config_file

Data type: String

The key-value file used by StoRM Info Provider as its configuration file.

info_sitename

Data type: String

It’s the human-readable name of your site used to set the Glue-SiteName attribute.

info_storage_default_root

Data type: String

Default directory for Storage Areas.

info_endpoint_quality_level

Data type: Integer

Endpoint maturity level to be published by the StoRM info provider. Default value: 2.

info_webdav_pool_list

Data type: Array[Storm::Backend::WebdavPoolMember]

List of published webdav endpoints.

info_frontend_host_list

Data type: Array[Storm::Backend::SrmPoolMember]

List of published srm endpoints.

jvm_options

Data type: String

jmx

Data type: Boolean

jmx_options

Data type: String

lcmaps_db_file

Data type: String

lcmaps_policy_name

Data type: String

lcmaps_log_file

Data type: String

lcmaps_debug_level

Data type: Integer

http_turl_prefix

Data type: String

storm_limit_nofile

Data type: Integer

Sets LimitNOFILE value.

manage_path_authz_db

Data type: Boolean

If true, allows to set content of path-authz.db file.

path_authz_db_file

Data type: String

If manage_path_authz_db is true, set the content from this source path

manage_storm_properties

Data type: Boolean

If true, allows to set content of storm.properties file wuth the file pointed by path_storm_properties.

path_storm_properties

Data type: String

If manage_storm_properties is true, set the content of storm.properties from this source path

storm::backend::config

StoRM Backend config class

storm::backend::configdb

StoRM Backend DB config class

storm::backend::install

StoRM Backend install class

storm::backend::service

StoRM Backend service class

storm::db

Install MariaDB server and client, create empty databases 'storm_db' and 'storm_be_ISAM', add storm user and all the necessary grants.

Examples

Basic usage:

include storm::db

Parameters

The following parameters are available in the storm::db class:

• fqdn_hostname
• root_password
• storm_username
• storm_password
• override_options
• limit_no_file

fqdn_hostname

Data type: String

The Fully Qualified Domain Name of the host. Default value got from Puppet fact fqdn.

Default value: $fqdn

root_password

Data type: String

MySQL root password. Default: 'storm'.

storm_username

Data type: String

The username of the user used by storm services to query the databases. Default 'storm'.

storm_password

Data type: String

The password of 'storm' username used by storm services to access the databases. Default: 'storm'.

override_options

Data type: Data

MySQL server override options. Read more about this at https://forge.puppet.com/puppetlabs/mysql/reference#override_options.

limit_no_file

Data type: Integer

MariaDB setting for limitNoFile

storm::frontend

StoRM Frontend puppet module

Examples

Example of usage

class { 'storm::frontend':
  be_xmlrpc_host  => 'storm-backend.host.org',
  be_xmlrpc_token => 'my-secret-xmlrpc-token',
  db_host         => 'storm-backend.host.org',
  db_passwd       => 'my-secret-db-password',
}

Parameters

The following parameters are available in the storm::frontend class:

• storm_frontend_server_conf_file
• be_xmlrpc_host
• be_xmlrpc_port
• be_xmlrpc_token
• be_xmlrpc_path
• be_recalltable_port
• db_host
• db_user
• db_passwd
• port
• threadpool_maxpending
• threadpool_threads_number
• gsoap_maxpending
• check_user_blacklisting
• argus_pepd_endpoint
• argus_resource_id
• monitoring_enabled
• monitoring_time_interval
• monitoring_detailed
• security_enable_vomscheck
• log_debuglevel

storm_frontend_server_conf_file

Data type: String

Use this parameter to provide your own storm-frontend-server.conf file. This can be used, for example, when you're using the latest module version to configure an old StoRM deployment. This parameter is used only when it has a non-empty value and it overwrites several other module parameters. Default value: empty string (that means not used).

be_xmlrpc_host

Data type: String

StoRM Backend hostname.

Default value: $fqdn

be_xmlrpc_port

Data type: Integer

StoRM Backend XML-RPC server port. Default value: 8080.

be_xmlrpc_token

Data type: String

Security token used for communicating with StoRM Backend. Mandatory.

be_xmlrpc_path

Data type: String

StoRM Backend XML-RPC server path. Default value: '/RPC2'.

be_recalltable_port

Data type: Integer

StoRM Backend REST server port running on the Backend machine. Default value: 9998.

db_host

Data type: String

Host for database connection.

Default value: $be_xmlrpc_host

db_user

Data type: String

User for database connection. Default value: 'storm'.

db_passwd

Data type: String

Password for database connection. Default value: 'storm'.

port

Data type: Integer

Frontend service port. Default value: 8444.

threadpool_maxpending

Data type: Integer

Size of the internal queue used to maintain SRM tasks in case there are no free worker threads. Default value: 200.

threadpool_threads_number

Data type: Integer

Size of the worker thread pool. Default value: 50.

gsoap_maxpending

Data type: Integer

Size of the GSOAP queue used to maintain pending SRM requests. Default value: 1000.

check_user_blacklisting

Data type: Boolean

Enable/disable user blacklisting. Default value: false.

argus_pepd_endpoint

Data type: String

The complete service endpoint of Argus PEP server. Mandatory if check_user_blacklisting is true.

argus_resource_id

Data type: String

The resource id is used to target a resource (or set of resources, if wildcards are used) under the control of Argus authorization. Mandatory if check_user_blacklisting is true.

monitoring_enabled

Data type: Boolean

Enable/disable monitoring. Default value: true.

monitoring_time_interval

Data type: Integer

Time interval in seconds between each monitoring round. Default value: 60.

monitoring_detailed

Data type: Boolean

Enable/disable detailed monitoring. Default value: false.

security_enable_vomscheck

Data type: Boolean

Flag to enable/disable checking proxy VOMS credentials. Default value: true.

log_debuglevel

Data type: String

Logging level. Possible values are: ERROR, WARN, INFO, DEBUG, DEBUG2. Default value: INFO.

storm::frontend::config

StoRM Frontend config class

storm::frontend::install

StoRM Frontend install class

storm::frontend::service

StoRM Frontend service class

storm::gridftp

Parameters

The StoRM GridFTP configuration parameters are:

• port: the port used by GridFTP server service;
• port_range: the range of ports used by transfer sockets; format is 'MIN,MAX';
• connections_max: the number of max allowed connections to server;

Examples

Example of usage

class { 'storm::gridftp':
  port            => 2811,
  port_range      => '20000,25000',
  connections_max => 2000,
}

Parameters

The following parameters are available in the storm::gridftp class:

• port
• port_range
• connections_max
• log_single
• log_transfer
• redirect_lcmaps_log
• llgt_log_file
• lcmaps_debug_level
• lcas_debug_level
• load_storm_dsi_module
• log_level
• data_interface

port

Data type: Integer

The port used by GridFTP server service.

port_range

Data type: String

The range of ports used by transfer sockets; format is 'MIN,MAX'.

connections_max

Data type: Integer

The number of max allowed connections to server.

log_single

Data type: String

Session log file path. Default is: /var/log/storm/storm-gridftp-session.log

log_transfer

Data type: String

Transfer log file path. Default is: /var/log/storm/storm-globus-gridftp.log

redirect_lcmaps_log

Data type: Boolean

If true, redirect the LCMAPS log to the file specified by 'llgt_log_file'.

llgt_log_file

Data type: String

The LCMAPS log file used if 'redirect_lcmaps_log' is true.

lcmaps_debug_level

Data type: Integer

The LCMAPS logging level. Values from 0 (ERROR) to 5 (DEBUG). Default: 3 (INFO)

lcas_debug_level

Data type: Integer

The LCAS logging level. Values from 0 (ERROR) to 5 (DEBUG). Default: 3 (INFO)

load_storm_dsi_module

Data type: Boolean

Enable/Disable StoRM DSI module. Default: true (enabled)

log_level

Data type: String

data_interface

Data type: Optional[String]

(Optional) Hostname or IP address of the interface to use for data connections. Set this to your external IP address in case your public address is different from your local address (e.g. the floating IP address of a Virtual Machine).

storm::gridftp::config

StoRM GridFTP config class

storm::gridftp::install

StoRM GridFTP install class

storm::gridftp::service

StoRM GridFTP service class

storm::repo

Choose which StoRM repository you want to intall and enable. Also a custom list of repository URL can be specified.

Examples

Install all the repositories and enable only nightly repo as follow:

class { 'storm::repo':
  enabled => ['stable'],
}

Parameters

The following parameters are available in the storm::repo class:

• installed
• enabled
• extra

installed

Data type: Array[Enum['stable', 'beta', 'nightly']]

The list of repositories that have to be installed. Allowed values are stable, beta and nightly. Optional.

enabled

Data type: Array[Enum['stable', 'beta', 'nightly']]

The list of repositories that have to be enabled. Allowed values are stable, beta and nightly. Optional.

extra

Data type: Array[Storm::CustomRepo]

A list of repository that have to be created. Optional.

storm::users

Parameters

StoRM needs a 'storm' Unix user, member of an 'edguser' group. This class creates all the necessary users.

Examples

Example of usage

class { 'storm::users':
  groups => {
    'infosys' => {
      gid => '996',
    },
  },
  users => {
    'edguser' => {
      'comment' => 'Edguser user',
      'groups'  => [ edguser, infosys, storm, ],
      'uid'     => '995',
      'gid'     => '995',
    },
    'storm' => {
      'comment' => 'StoRM user',
      'groups'  => [ storm, edguser, ],
      'uid'     => '991',
      'gid'     => '991',
    },
  }
}

Parameters

The following parameters are available in the storm::users class:

• groups
• users

groups

Data type: Accounts::Group::Hash

users

Data type: Accounts::User::Hash

storm::webdav

StoRM WebDAV puppet module

Examples

Basic usage

class { 'storm::webdav':
  storage_areas => [
    {
      name                       => 'test.vo',
      root_path                  => '/storage/test.vo',
    },
  ],
}

Parameters

The following parameters are available in the storm::webdav class:

• storage_areas
• storage_areas_source_directory
• hostnames
• http_port
• https_port
• trust_anchors_refresh_interval
• max_concurrent_connections
• max_queue_size
• connector_max_idle_time
• vo_map_files_enable
• vo_map_files_config_dir
• vo_map_files_refresh_interval
• tpc_max_connections
• tpc_max_connections_per_route
• tpc_verify_checksum
• tpc_timeout_in_secs
• tpc_tls_protocol
• tpc_report_delay_secs
• tpc_enable_tls_client_auth
• tpc_progress_report_thread_pool_size
• tpc_enable_expect_continue_threshold
• jvm_opts
• authz_server_enable
• authz_server_issuer
• authz_server_max_token_lifetime_sec
• authz_server_secret
• require_client_cert
• use_conscrypt
• tpc_use_conscrypt
• enable_http2
• debug
• debug_port
• debug_suspend
• storm_limit_nofile
• user
• tape_wellknown_source
• tls_protocol
• checksum_filter_enabled
• macaroon_filter_enabled
• file_buffer_size
• voms_trust_store_dir
• voms_trust_store_refresh_interval_sec
• voms_cache_enabled
• voms_cache_entry_lifetime_sec
• scitags_enabled
• scitags_collector
• nginx_enabled

storage_areas

Data type: Array[Storm::Webdav::StorageArea]

List of storage area's configuration. Default value: empty list.

storage_areas_source_directory

Data type: Optional[String]

Source directory from where all properties files are copied into WebDAV sa.d folder. Optional.

hostnames

Data type: Array[String]

Sets STORMWEBDAV_HOSTNAME(N) environment variables.

http_port

Data type: Integer

Sets STORM_WEBDAV_HTTP_PORT environment variable.

https_port

Data type: Integer

Sets STORM_WEBDAV_HTTPS_PORT environment variable.

trust_anchors_refresh_interval

Data type: Integer

Sets STORM_WEBDAV_TRUST_ANCHORS_REFRESH_INTERVAL environment variable.

max_concurrent_connections

Data type: Integer

Sets STORM_WEBDAV_MAX_CONNECTIONS environment variable.

max_queue_size

Data type: Integer

Sets STORM_WEBDAV_MAX_QUEUE_SIZE environment variable.

connector_max_idle_time

Data type: Integer

Sets STORM_WEBDAV_CONNECTOR_MAX_IDLE_TIME environment variable.

vo_map_files_enable

Data type: Boolean

Sets STORM_WEBDAV_VO_MAP_FILES_ENABLE environment variable.

vo_map_files_config_dir

Data type: String

Sets STORM_WEBDAV_VO_MAP_FILES_CONFIG_DIR environment variable.

vo_map_files_refresh_interval

Data type: Integer

Sets STORM_WEBDAV_VO_MAP_FILES_REFRESH_INTERVAL environment variable.

tpc_max_connections

Data type: Integer

Sets STORM_WEBDAV_TPC_MAX_CONNECTIONS environment variable.

tpc_max_connections_per_route

Data type: Integer

Sets STORM_WEBDAV_TPC_MAX_CONNECTIONS_PER_ROUTE environment variable.

tpc_verify_checksum

Data type: Boolean

Sets STORM_WEBDAV_TPC_VERIFY_CHECKSUM environment variable.

tpc_timeout_in_secs

Data type: Integer

Sets STORM_WEBDAV_TPC_TIMEOUT_IN_SECS environment variable.

tpc_tls_protocol

Data type: String

Sets STORM_WEBDAV_TPC_TLS_PROTOCOL environment variable.

tpc_report_delay_secs

Data type: Integer

Sets STORM_WEBDAV_TPC_REPORT_DELAY_SECS environment variable.

tpc_enable_tls_client_auth

Data type: Boolean

Sets STORM_WEBDAV_TPC_ENABLE_TLS_CLIENT_AUTH environment variable.

tpc_progress_report_thread_pool_size

Data type: Integer

Sets STORM_WEBDAV_TPC_PROGRESS_REPORT_THREAD_POOL_SIZE environment variable.

tpc_enable_expect_continue_threshold

Data type: Integer

Sets STORM_WEBDAV_TPC_ENABLE_EXPECT_CONTINUE_THRESHOLD environment variable.

jvm_opts

Data type: String

Sets part of STORM_WEBDAV_JVM_OPTS environment variable.

authz_server_enable

Data type: Boolean

Sets STORM_WEBDAV_AUTHZ_SERVER_ENABLE environment variable.

authz_server_issuer

Data type: String

Sets STORM_WEBDAV_AUTHZ_SERVER_ISSUER environment variable if authz_server_enable is true.

authz_server_max_token_lifetime_sec

Data type: Integer

Sets STORM_WEBDAV_AUTHZ_SERVER_MAX_TOKEN_LIFETIME_SEC environment variable if authz_server_enable is true.

authz_server_secret

Data type: String

Sets STORM_WEBDAV_AUTHZ_SERVER_SECRET environment variable if authz_server_enable is true.

require_client_cert

Data type: Boolean

Sets STORM_WEBDAV_REQUIRE_CLIENT_CERT environment variable.

use_conscrypt

Data type: Boolean

Sets STORM_WEBDAV_USE_CONSCRYPT environment variable.

tpc_use_conscrypt

Data type: Boolean

Sets STORM_WEBDAV_TPC_USE_CONSCRYPT environment variable.

enable_http2

Data type: Boolean

Sets STORM_WEBDAV_ENABLE_HTTP2 environment variable.

debug

Data type: Boolean

Sets part of STORM_WEBDAV_JVM_OPTS environment variable. It enables remote debug.

debug_port

Data type: Integer

Sets part of STORM_WEBDAV_JVM_OPTS environment variable. It sets the remote debug port if remote debug is enabled.

debug_suspend

Data type: Boolean

Sets part of STORM_WEBDAV_JVM_OPTS environment variable. It sets debug suspend value in case remote debug is enabled.

storm_limit_nofile

Data type: Integer

Sets LimitNOFILE value.

user

Data type: String

Sets Unix user which runs the service

tape_wellknown_source

Data type: String

Sets the absolute path of the source file containing the tape REST API well-known endpoints

tls_protocol

Data type: String

Sets TLS protocol for non-TPC requests

checksum_filter_enabled

Data type: Boolean

Enable checksum filter which adds checksum as an header following RFC 3230.

macaroon_filter_enabled

Data type: Boolean

Enable Macaroon filter to process Macaroon tokens. Requires authz server enabled.

file_buffer_size

Data type: Integer

Buffer size for both internal and third-party copy requests. Valid values are numbers greater/equals than 4096.

voms_trust_store_dir

Data type: String

VOMS Trust Store directory

voms_trust_store_refresh_interval_sec

Data type: Integer

VOMS Trust Store refresh interval

voms_cache_enabled

Data type: Boolean

Enable caching for VOMS certificate validation

voms_cache_entry_lifetime_sec

Data type: Integer

Cache entries lifetime, used if caching for VOMS certificate validation is enabled

scitags_enabled

Data type: Boolean

Enable SciTags flow marking

scitags_collector

Data type: Optional[String]

Remote address of SciTags collector. Optional.

nginx_enabled

Data type: Boolean

Enable nginx support

storm::webdav::config

StoRM WebDAV config class

storm::webdav::install

StoRM WebDAV install class

storm::webdav::service

StoRM WebDAV service class

Defined types

storm::backend::service_conf_file

Starting from Puppet module v2.0.0, site administrators can inject one or more .conf files into /etc/systemd/system/storm-backend-server.service.d directory.

Examples

class { 'storm::backend':
  hostname => 'storm-backend.test.example',
  # ...
}
storm::backend::service_conf_file { 'override.conf':
  source => '/path/to/my/override.conf',
}

Parameters

The following parameters are available in the storm::backend::service_conf_file defined type:

• source

source

Data type: String

The source of file resource. It can be an absolute path or a Puppet module relative path.

storm::backend::storage_site_report

  by using this defined type.

Examples

class { 'storm::backend':
  hostname => 'storm-backend.test.example',
  # ...
}
storm::backend::storage_site_report { 'storage-site-report':
  report_path => '/storage/info/report.json', # the internal storage area path
  minute      => '*/20', # set cron's minute
}

Parameters

The following parameters are available in the storm::backend::storage_site_report defined type:

• report_path
• minute

report_path

Data type: String

The full path of the generated report. Usually it points to a "info" storage area.

minute

Data type: String

The cron job's minute parameter. Refer to Resource Type cron.

Default value: '*/30'

storm::rootdir

StoRM main storage area root directory defined resource

Parameters

The following parameters are available in the storm::rootdir defined type:

• mode

mode

Data type: String

Default value: '0755'

storm::sarootdir

StoRM Storage Area root directory defined resource

Parameters

The following parameters are available in the storm::sarootdir defined type:

• mode

mode

Data type: String

Default value: '0750'

storm::webdav::application_file

Starting from Puppet module v2.0.0, the management of application.yml file has been removed from storm::webdav class. Site administrators can edit their own configuration files or use this defined type to inject one or more YAML files into the proper directory.

Examples

class { 'storm::webdav':
  hostnames => ['storm-webdav.test.example', 'alias-for-storm-webdav.test.example'],
}
storm::webdav::application_file { 'application.yml':
  source => '/path/to/my/application.yml',
}
storm::webdav::application_file { 'application-wlcg.yml':
  source => '/path/to/my/application-wlcg.yml',
}

Parameters

The following parameters are available in the storm::webdav::application_file defined type:

• source

source

Data type: String

The source of file resource. It can be an absolute path or a Puppet module relative path.

storm::webdav::drop_in_file

Use this define to inject one or more .conf files into /etc/systemd/system/storm-webdav.service.d directory.

Examples

class { 'storm::webdav':
  # storm webdav parameters
}
storm::webdav::drop_in_file { 'override.conf':
  source => '/path/to/my/override.conf',
}

Parameters

The following parameters are available in the storm::webdav::drop_in_file defined type:

• source

source

Data type: String

The source of file resource. It can be an absolute path or a Puppet module relative path.

storm::webdav::storage_area_file

Storage Areas can be configured singularly by using this defined type. This strategy allows site administrators to keep their manifests unaware of the improvements on StoRM WebDAV code. For example, if a new property is added into Storage Area configuration files, you haven't to update your Puppet module and all the service configuration will continue working.

Examples

class { 'storm::webdav':
  hostnames => ['storm-webdav.test.example', 'alias-for-storm-webdav.test.example'],
}
storm::webdav::storage_area_file { 'test.vo.properties':
  source => '/path/to/my/test.vo.properties',
}
storm::webdav::storage_area_file { 'test.vo.2.properties':
  source => '/path/to/my/test.vo.2.properties',
}

Parameters

The following parameters are available in the storm::webdav::storage_area_file defined type:

• source

source

Data type: String

The source of file resource. It can be an absolute path or a Puppet module relative path.

Data types

Storm::Backend::Acl

The ACL type for storm-backend-server

Alias of

Struct[{
  group => String,
  permission => Enum['R', 'W', 'RW'],
}]

Storm::Backend::AclMode

The AclMode type for storm-backend-server

Alias of Enum['AoT', 'JiT']

Storm::Backend::BalanceStrategy

The BalanceStrategy type for storm-backend-server

Alias of Enum['round-robin', 'smart-rr', 'random', 'weight']

Storm::Backend::Endpoint

The Endpoint type for storm-backend-server

Alias of

Struct[{
  schema   => Enum['rfio','xroot','gsiftp','srm','http','https'],
  hostname => String,
  port     => Integer,
  path     => String,
}]

Storm::Backend::FileSystem

The FileSystem type for storm-backend-server

Alias of

Struct[{
  type => Enum['ext3', 'gpfs'],
  driver => Enum['posixfs', 'gpfs', 'test'],
  space_system => Optional[Enum['MockSpaceSystem', 'GPFSSpaceSystem']],
}]

Storm::Backend::FsDriver

The FsDriver type for storm-backend-server

Alias of Enum['posixfs', 'gpfs', 'test']

Storm::Backend::FsType

The FsType type for storm-backend-server

Alias of Enum['posixfs', 'gpfs', 'test']

Storm::Backend::Gsiftp

The Gsiftp type for storm-backend-server

Alias of

Struct[{
  pool => Storm::Backend::Pool,
}]

Storm::Backend::GsiftpPoolMember

The GsiftpPoolMember type for storm-backend-server

Alias of

Struct[{
  hostname => String,
  port     => Optional[Integer],
  weight   => Optional[Integer],
}]

Storm::Backend::Pool

The Pool type for storm-backend-server

Alias of

Struct[{
  balance_strategy => Optional[Enum['round-robin', 'smart-rr', 'random', 'weight']],
  members          => Array[Struct[{
    hostname => String,
    port     => Optional[Integer],
    weigth   => Optional[Integer],
  }]],
}]

Storm::Backend::Quota

The Quota type for storm-backend-server

Alias of

Struct[{
  device => String,
  type   => Enum['username', 'group', 'fileset'],
  value  => String,
}]

Storm::Backend::Rfio

The Rfio type for storm-backend-server

Alias of

Struct[{
  hostname => String,
  port => Optional[Integer],
}]

Storm::Backend::SrmPoolMember

The SrmPoolMember type for storm-backend-server

Alias of

Struct[{
  hostname => String,
  port     => Optional[Integer],
}]

Storm::Backend::StorageArea

The storage area type for storm-backend-server

Alias of

Struct[{
    name                         => String,
    root_path                    => String,
    access_points                => Array[String],
    vos                          => Array[String],
    fs_type                      => Optional[Storm::Backend::FsType],
    space_token                  => Optional[String],
    authz                        => Optional[String],
    storage_class                => Optional[Storm::Backend::StorageClass],
    online_size                  => Integer,
    nearline_size                => Optional[Integer],
    acl_mode                     => Optional[Storm::Backend::AclMode],
    default_acl_list             => Optional[Array[Storm::Backend::Acl]],
    quota                        => Optional[Storm::Backend::Quota],
    dn_regex                     => Optional[String],
    anonymous_http_read          => Optional[Boolean],
    transfer_protocols           => Optional[Array[Storm::Backend::TransferProtocol]],
    rfio_hostname                => Optional[String],
    rfio_port                    => Optional[Integer],
    xroot_hostname               => Optional[String],
    xroot_port                   => Optional[Integer],
    gsiftp_pool_balance_strategy => Optional[Storm::Backend::BalanceStrategy],
    gsiftp_pool_members          => Optional[Array[Storm::Backend::GsiftpPoolMember]],
    webdav_pool_balance_strategy => Optional[Storm::Backend::BalanceStrategy],
    webdav_pool_members          => Optional[Array[Storm::Backend::WebdavPoolMember]],
}]

Storm::Backend::StorageClass

The StorageClass type for storm-backend-server

Alias of Enum['T0D1', 'T1D0', 'T1D1']

Storm::Backend::TransferProtocol

The TransferProtocol type for storm-backend-server

Alias of Enum['file', 'gsiftp', 'xroot', 'webdav']

Storm::Backend::Webdav

The WebDAV type for storm-backend-server

Alias of

Struct[{
  pool => Array[Struct[{
    hostname => String,
    port     => Optional[Integer],
  }]],
}]

Storm::Backend::WebdavPoolMember

The WebdavPoolMember type for storm-backend-server

Alias of

Struct[{
  hostname => String,
  http_port => Optional[Integer],
  https_port => Optional[Integer],
}]

Storm::Backend::Xroot

The Xroot type for storm-backend-server

Alias of

Struct[{
  hostname => String,
  port => Optional[Integer],
}]

Storm::CustomRepo

The storage area type for storm-webdav

Alias of

Struct[{
  name    => String,
  baseurl => String,
}]

Storm::Webdav::StorageArea

The storage area type for storm-webdav.

Alias of

Struct[{
  name                           => String,
  root_path                      => String,
  filesystem_type                => Optional[String],
  access_points                  => Optional[Array[String]],
  vos                            => Optional[Array[String]],
  orgs                           => Optional[Array[String]],
  authenticated_read_enabled     => Optional[Boolean],
  anonymous_read_enabled         => Optional[Boolean],
  vo_map_enabled                 => Optional[Boolean],
  vo_map_grants_write_permission => Optional[Boolean],
  orgs_grant_read_permission     => Optional[Boolean],
  orgs_grant_write_permission    => Optional[Boolean],
  wlcg_scope_authz_enabled       => Optional[Boolean],
  fine_grained_authz_enabled     => Optional[Boolean],
}]

Changelog

All notable changes to this project will be documented in this file.

[4.4.0]

• Add storm::webdav::nginx_enabled parameter to WebDAV that enables the forwarding of GET requests to an external NGINX

[4.3.1]

• Add notify on storage area source directory (StoRM WebDAV)

[4.3.0]

• Add SciTags collector parameter storm::webdav::scitags_collector
• Update SciTags flow registry URL

[4.2.0]

• Added the opportunity to specify an external source for WebDAV storage area directory. Use storm::webdav::storage_areas_source_directory. This parameter, if set, overrides storm::webdav::storage_areas.
• Enable SciTags support with storm::webdav::scitags_enabled.

[4.1.0]

• Added the opportunity to specify an external source for Backend's storm.properties file. Use storm::backend::manage_storm_properties and storm::backend::path_storm_properties.

[4.0.1]

• Added new StoRM WebDAV parameters for the related environment variables:
  • storm::webdav::tls_protocol
  • storm::webdav::checksum_filter_enabled
  • storm::webdav::macaroon_filter_enabled
  • storm::webdav::file_buffer_size
  • storm::webdav::voms_trust_store_dir
  • storm::webdav::voms_trust_store_refresh_interval_sec
  • storm::webdav::voms_cache_enabled
  • storm::webdav::voms_cache_entry_lifetime_sec

[4.0.0]

• Puppet version >= 7.0.0 is required
• Added storm::gridftp::data_interface parameter in order to support a configuration behind a public IP
• Added storm::gridftp::log_level parameter in order to allow logging level
• Removed storm::webdav::ensure_empty_storage_area_dir: storage area directory is now automatically purged of not managed properties files
• Added storm::webdav::tpc_enable_expect_continue_threshold in order to set a threshold from which an header with Expect: 100 continue is added
• Added storm::webdav::user parameter in order to allow the setting of a custom user used to run the service
• Added storm::webdav::tape_wellknown_source parameter in order to specify which JSON file has to be loaded as content of the well-known endpoint for WLCG Tape REST API implementation
• Within update-site-report script, JSON file is now moved and not copied to avoid the growth of useless files in tmp directory
• Removed storm::backend::debug* parameters
• Removed storm::backend::install_native_libs_gpfs parameter because the installation of the GPFS libs is done if at least one storage area has fs_type GPFS.
• Added a retry mechanism if StoRM Info Provider configure command fails
• Removed the custom management of systemctl daemon reload because now it's supported by Puppet 7

[3.4.0]

• Removed useless storm::frontend parameters gridmap_dir and gridmap_file

[3.3.2]

• Changed default value to an empty array for vos field of Storm::Webdav::StorageArea
• Added pool of webdav endpoints into backend's namespace.xml

[3.3.1]

• Added missing Argus's resource-id parameter within Frontend's configuration.
• Added support for using an external file for StoRM Frontend's configuration file.
• Removed storm::frontend::security_enable_mapping parameter.
• The storm::frontend::be_xmlrpc_host value is initialized with local FQDN.

[3.3.0]

• Added missing StoRM WebDAV configuration parameters
• StoRM WebDAV hostname list initialized with local FQDN
• Added storm::webdav::drop_in_file define in order to allow adding custom drop-in files to be stored in /etc/systemd/system/storm-webdav.service.d
• Added storm::webdav::tpc_max_connections_per_route, storm::webdav::tpc_timeout_in_secs, storm::webdav::tpc_tls_protocol, storm::webdav::tpc_report_delay_secs, storm::webdav::tpc_enable_tls_client_auth, storm::webdav::tpc_progress_report_thread_pool_size parameters.
• Removed storm::webdav::manage_storage_areas parameter. The logic has been simplified to managing storage areas only if storm::webdav::storage_areas list is defined.
• Added storm::webdav::ensure_empty_storage_area_dir parameter to ensure a cleaned storage area directory before adding the .properties files.
• Increased StoRM WebDAV default heap size to 1024m and default timeout for TPC to 30 seconds

[3.2.1]

• Fixed permissions on info-provider configuration file

[3.2.0]

• Added daemon reload on backend and webdav restart command

[3.1.1]

• Improved README. No changes on module.

[3.1.0]

• Added mode parameter to storm::rootdir and storm::sarootdir defined types
• Increased default value for wait_timeout in storm::db::override_options

[3.0.0]

• Converted to pdk project
• Removed storm::storage class
• Added storm::sarootdir defined type
• Use hiera YAML files for all default parameters
• Removed all params.pp files

[2.2.1] - 2020-11-09

• Fixed type into data file

[2.2.0] - 2020-11-09

• Moved default parameters into YAML data files
• Deleted params.pp files
• Added hiera.yaml

[2.1.0] - 2020-10-29

• Moved gridftp.conf into /etc/grid-security directory

[2.0.1] - 2020-10-19

• Fixed CHANGELOG
• Improved documentation.

[2.0.0] - 2020-10-19

• Added defined types for application-*.yml and storage area files used by WebDAV class.
• Removed direct support for application.yml template file.
• Added defined types for service conf files of Backend.
• Added defined type for storage site reporting.
• Added tests.
• Removed docker deploy image.

[1.1.1] - 2020-10-06

• Removed log directory management

[1.1.0] - 2020-10-05

• Removed storm::mapping class

[1.0.5] - 2020-10-02

• Added path-authz.db support to StoRM Backend

[1.0.0] - 2020-09-25

• Moved database creation and MySQL/MariaDB installation on a separated class storm::db
• Backend class only import SQL script of databases, only if database or table version not exists
• WebDAV class by default doesn't manage application.yml file. It can be optionally managed from a source or from variables.
• WebDAV class by default manage storage areas. A source directory or the array of storage areas can be provided.
• Mapping class also manages VOMS roles and multiple groups.
• Removed epel and umd repo from repo class
• Removed testca and testvos classes

[0.2.3] - 2020-05-25

Changed

• Allow enable/disable StoRM DSI module for gridftp

[0.2.1] - 2020-05-25

Changed

• Updated README.md

[0.2.0] - 2020-05-15

Added

• Support for StoRM Frontend service configuration

Changed

• Fixed ignored WebDAV HTTP2 and Conscrypt options
• Allow redirection of LCMAPS logging to a particular file
• Fixed conflict on storm user/group creation when the same node install WebDAV and Frontend services

[0.1.0] - 2019-10-09

Added

• Support for StoRM WebDAV service configuration
• Support for StoRM GridFTP server configuration
• Documentation at REFERENCE.md
• Documentation at https://italiangrid.github.io/storm-puppet-module