clickhouse_keeper

pdk

A module to manage ClickHouse Keeper cluster

Project URL RSS Feed Report issues

Module Stats

1,528 downloads

841 latest version

5.0 quality score

Version information

released Jun 21st 2024

This version is compatible with:

Puppet Enterprise 2025.4.x, 2025.3.x, 2025.2.x, 2025.1.x, 2023.8.x, 2023.7.x, 2023.6.x, 2023.5.x, 2023.4.x, 2023.3.x, 2023.2.x, 2023.1.x, 2023.0.x, 2021.7.x, 2021.6.x, 2021.5.x, 2021.4.x, 2021.3.x, 2021.2.x, 2021.1.x, 2021.0.x, 2019.8.x
Puppet >= 6.21.0 < 9.0.0
CentOS
,
OracleLinux
,
RedHat
,
Scientific
,
Debian
,
Ubuntu

Start using this module

Installation method

Add this module to your Puppetfile:

mod 'deric-clickhouse_keeper', '1.0.0'

Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add deric-clickhouse_keeper

Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install deric-clickhouse_keeper --version 1.0.0

Documentation

deric/clickhouse_keeper — version 1.0.0 Jun 21st 2024

Puppet ClickHouse Keeper

A Puppet module to manage ClickHouse Keeper installation. ClickHouse Keeper provides distributed key-value storage with API taht is compatible with ZooKeeper.

Unlike ZooKeeper ClickHouse Keeper is written in C++ and uses the RAFT algorithm implementation. This algorithm allows linearizability for reads and writes, and has several open-source implementations in different languages.

Usage

include clickhouse_keeper

And you might want to modify configuration for incoming client connections, ClickHouse Keeper tcp_port defaults to 9181.

clickhouse_keeper::tcp_port: 2181
clickhouse_keeper::listen_host: 0.0.0.0
clickhouse_keeper::enable_ipv6: false
clickhouse_keeper::address: "%{facts.fqdn}"

Parameters

address Exported address to raft config, used only when export_raft is true
id Must be unique among servers
cluster Used to distinguish multiple clusters when configuration is exported
manage_config
manage_repo Whether APT/RPM repository should be managed by Puppet
manage_package
manage_user
manage_service
export_raft
raft_config
raft_port Port for internal communication between ClickHouse Keeper nodes
generate_certs
packages OS packages to be installed
package_ensure
package_install_options
owner System user account to own config files
group System group to own config files
config_dir Path to config directory
config_file
log_level
raft_log_level
log_file
error_file
log_size
log_count
listen_host Bind address for client connections
enable_ipv6 Whether IPv6 address should be binded (default: true).
max_connections default: 4096
service_name
service_enable
service_ensure
tcp_port Port for client connections
tcp_port_secure SSL port, requires valid certificates
certificate Path to public ssl certificate
private_key Path to private ssl certificate key
dhparams Path to DH params file
prometheus_port If defined metrics will be exposed at given port and /metrics endpoint
log_storage_path Keeper coordination logs (raft)
snapshot_storage_path Snapshots path
operation_timeout
min_session_timeout
session_timeout

Acceptance tests

BEAKER_destroy=no BEAKER_setfile=debian10-64 bundle exec rake beaker

Reference

Classes

Public Classes

clickhouse_keeper: Manages Clickouse Keeper
clickhouse_keeper::repo: Installs repository for Clickhouse.

Private Classes

clickhouse_keeper::config: Manage Clickouse keeper config

Defined types

Private Defined types

clickhouse_keeper::raft: A member of raft cluster

Classes

`clickhouse_keeper`

Manages Clickouse Keeper

Examples

include clickhouse_keeper

Parameters

The following parameters are available in the clickhouse_keeper class:

address
id
cluster
manage_config
manage_repo
manage_package
manage_user
manage_service
export_raft
raft_config
raft_port
generate_certs
packages
package_ensure
package_install_options
owner
group
config_dir
config_file
log_level
raft_log_level
log_file
error_file
log_size
log_count
listen_host
enable_ipv6
max_connections
service_name
service_enable
service_ensure
tcp_port
tcp_port_secure
certificate
private_key
dhparams
prometheus_port
log_storage_path
snapshot_storage_path
keeper_dir
operation_timeout
min_session_timeout
session_timeout

`address`

Data type: String

Exported address to raft config, used only when export_raft is true

Default value: $facts['networking']['ip']

`id`

Data type: Integer

Must be unique among servers

Default value: fqdn_rand(255, $facts['networking']['ip'])

`cluster`

Data type: String

Default value: 'main'

`manage_config`

Data type: Boolean

Default value: true

`manage_repo`

Data type: Boolean

Whether APT/RPM repository should be managed by Puppet

Default value: true

`manage_package`

Data type: Boolean

Default value: true

`manage_user`

Data type: Boolean

Default value: true

`manage_service`

Data type: Boolean

Default value: true

`export_raft`

Data type: Boolean

Default value: true

`raft_config`

Data type: Clickhouse_Keeper::Raft_config

Default value: {}

`raft_port`

Data type: Integer

Default value: 9234

`generate_certs`

Data type: Boolean

Default value: false

`packages`

Data type: Array[String[1]]

OS packages to be installed

Default value: ['clickhouse-keeper']

`package_ensure`

Data type: String

Default value: 'present'

`package_install_options`

Data type: Array[String]

Default value: []

`owner`

Data type: String

System user account to own config files

Default value: 'clickhouse'

`group`

Data type: String

System group to own config files

Default value: 'clickhouse'

`config_dir`

Data type: Stdlib::AbsolutePath

Path to config directory

Default value: '/etc/clickhouse-keeper'

`config_file`

Data type: String

Default value: 'keeper_config.xml'

`log_level`

Data type: Clickhouse_Keeper::LogLevel

Default value: 'information'

`raft_log_level`

Data type: Clickhouse_Keeper::LogLevel

Default value: 'information'

`log_file`

Data type: Stdlib::AbsolutePath

Default value: '/var/log/clickhouse-keeper/clickhouse-keeper.log'

`error_file`

Data type: Stdlib::AbsolutePath

Default value: '/var/log/clickhouse-keeper/clickhouse-keeper.err.log'

`log_size`

Data type: String

Default value: '1000M'

`log_count`

Data type: Integer

Default value: 10

`listen_host`

Data type: String

Bind address for client connections

Default value: '127.0.0.1'

`enable_ipv6`

Data type: Boolean

Default value: true

`max_connections`

Data type: Integer

Default value: 4096

`service_name`

Data type: String

Default value: 'clickhouse-keeper'

`service_enable`

Data type: Boolean

Default value: true

`service_ensure`

Data type: String

Default value: 'running'

`tcp_port`

Data type: Integer

Port for client connections

Default value: 9181

`tcp_port_secure`

Data type: Optional[Integer]

Default value: undef

`certificate`

Data type: Stdlib::AbsolutePath

Default value: '/etc/clickhouse-keeper/server.crt'

`private_key`

Data type: Stdlib::AbsolutePath

Default value: '/etc/clickhouse-keeper/server.key'

`dhparams`

Data type: Stdlib::AbsolutePath

Default value: '/etc/clickhouse-keeper/dhparam.pem'

`prometheus_port`

Data type: Optional[Integer]

If defined metrics will be exposed at given port and /metrics endpoint

Default value: undef

`log_storage_path`

Data type: Stdlib::AbsolutePath

Keeper coordination logs (raft)

Default value: '/var/lib/clickhouse/coordination/logs'

`snapshot_storage_path`

Data type: Stdlib::AbsolutePath

Snapshots path

Default value: '/var/lib/clickhouse/coordination/snapshots'

`keeper_dir`

Data type: Stdlib::AbsolutePath

Default value: '/var/lib/clickhouse-keeper'

`operation_timeout`

Data type: Integer

Default value: 10000

`min_session_timeout`

Data type: Integer

Default value: 10000

`session_timeout`

Data type: Integer

Default value: 100000

`clickhouse_keeper::repo`

Installs repository for Clickhouse.

Examples

include clickhouse::repo

Data types

`Clickhouse_Keeper::LogLevel`

The Clickhouse_Keeper::LogLevel data type.

Alias of Enum['none', 'fatal', 'critical', 'error', 'warning', 'notice', 'information', 'debug', 'trace']

`Clickhouse_Keeper::Raft_config`

The Clickhouse_Keeper::Raft_config data type.

Alias of

Hash[String, Struct[{
      id => Integer,
      address => String,
      port => Integer,
      cluster => String,
  }]]

clickhouse_keeper

Version information

This version is compatible with:

Start using this module

Documentation

Puppet ClickHouse Keeper

Usage

Parameters

Acceptance tests

Reference

Table of Contents

Classes

Public Classes

Private Classes

Defined types

Private Defined types

Data types

Classes

clickhouse_keeper

Examples

Parameters

address

id

cluster

manage_config

manage_repo

manage_package

manage_user

manage_service

export_raft

raft_config

raft_port

generate_certs

packages

package_ensure

package_install_options

owner

group

config_dir

config_file

log_level

raft_log_level

log_file

error_file

log_size

log_count

listen_host

enable_ipv6

max_connections

service_name

service_enable

service_ensure

tcp_port

tcp_port_secure

certificate

private_key

dhparams

prometheus_port

log_storage_path

snapshot_storage_path

keeper_dir

operation_timeout

min_session_timeout

session_timeout

clickhouse_keeper::repo

Examples

Data types

Clickhouse_Keeper::LogLevel

Clickhouse_Keeper::Raft_config

Changelog

Release 1.0.0 (2024-06-21)

Release 0.3.0 (2024-06-06)

Release 0.2.0 (2024-02-03)

Release 0.1.0

Dependencies

PDK

`clickhouse_keeper`

`address`

`id`

`cluster`

`manage_config`

`manage_repo`

`manage_package`

`manage_user`

`manage_service`

`export_raft`

`raft_config`

`raft_port`

`generate_certs`

`packages`

`package_ensure`

`package_install_options`

`owner`

`group`

`config_dir`

`config_file`

`log_level`

`raft_log_level`

`log_file`

`error_file`

`log_size`

`log_count`

`listen_host`

`enable_ipv6`

`max_connections`

`service_name`

`service_enable`

`service_ensure`

`tcp_port`

`tcp_port_secure`

`certificate`

`private_key`

`dhparams`

`prometheus_port`

`log_storage_path`

`snapshot_storage_path`

`keeper_dir`

`operation_timeout`

`min_session_timeout`

`session_timeout`

`clickhouse_keeper::repo`

`Clickhouse_Keeper::LogLevel`

`Clickhouse_Keeper::Raft_config`