Forge Home

Modules

Resources

PuppetEcosystemPuppet CommunityUpdates

clickhouse

Installs, configures and manages Clickhouse server and client
Project URLRSS FeedReport issues

15,153 downloads

14,525 latest version

5.0 quality score

We run a couple of automated
scans to help you access a
module's quality. Each module is
given a score based on how well
the author has formatted their
code and documentation and
modules are also checked for
malware using VirusTotal.

Please note, the information below
is for guidance only and neither of
these methods should be considered
an endorsement by Puppet.

Version information

  • 1.0.6 (latest)
  • 1.0.5
  • 1.0.4
  • 1.0.3
released Jan 30th 2019
This version is compatible with:
  • Puppet Enterprise 2019.8.x, 2019.7.x, 2019.5.x, 2019.4.x, 2019.3.x, 2019.2.x, 2019.1.x, 2019.0.x, 2018.1.x, 2017.3.x, 2017.2.x, 2016.4.x
  • Puppet >= 4.10.0 < 7.0.0
  • , , ,

Start using this module

  • r10k or Code Manager
  • Bolt
  • Manual installation
  • Direct download

Add this module to your Puppetfile:

mod 'mfedotov-clickhouse', '1.0.6'
Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add mfedotov-clickhouse
Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install mfedotov-clickhouse --version 1.0.6

Direct download is not typically how you would use a Puppet module to manage your infrastructure, but you may want to download the module in order to inspect the code.

Download

Documentation

mfedotov/clickhouse — version 1.0.6 Jan 30th 2019

Build Status

clickhouse

Table of Contents

Description

The clickhouse module installs, configures and manages the Clickhouse Server service.

It also allows to create and manage users, quotas, profiles, dictionaries, databases as well as configure replication and sharding.

Setup

Setup Requirements

This module requires xml-simple gem, which is used to translate Hash configuration to Clickhouse XML format configuration files. To install it you need to execute following command on your puppetmaster server:

sudo puppetserver gem install xml-simple

Beginning with clickhouse

To install a server with the default options:

include 'clickhouse::server'.

To customize Clickhouse server options, you must also pass in an override hash:

class { 'clickhouse::server':
  override_options => $override_options
}

See Customize Server Options below for examples of the hash structure for $override_options.

Usage

All server configuration is done via clickhouse::server. To install client separatly, use clickhouse::client

Customize server options

To define server options, pass a hash structure of overrides in clickhouse::server. Server configuration parameters can be found at https://clickhouse.yandex/docs/en/operations/server_settings/settings/

$override_options = {
  parent_xml_tag => {
    item => thing,
  }
}

For example, to configure zstd compression pass following override options hash:

$override_options = {
  compression => {
    case => {
      min_part_size       => 10000000000,
      min_part_size_ratio => 0.01,
      method              => 'zstd',
    }
  }
}

It will add following section to Clickhouse configuration file:

<compression>
    <case>
        <min_part_size>10000000000</min_part_size>
        <min_part_size_ratio>0.01</min_part_size_ratio>
        <method>zstd</method>
    </case>
</compression>

Create a database

To create a database:

clickhouse_database { 'my_database':
  ensure => 'present',
}

Specify passwords for users

In addition to passing passwords in plain text, you can input them in sha256 encoding. For example:

class { 'clickhouse::server':
  users => {
    myuser => {
      password => '02472d6a1e23f73b37481bbd67949a5d16cbaf3d71770696f20a0cd773a2e682',
    }
  }
}

Install Clickhouse Server

This example shows how to install Clickhouse Server, configure zstd compression for it, create production profile and quota, create user alice and assign production profile and quota to her and also configure replication and three different sharding schemas: replicated - just a basic replication with single shard and two replicas; segmented - two shards without replicas; segmented_replicated - two shards each having two replicas:

class { 'clickhouse::server':
  override_options => {
    compression => {
      case => {
        min_part_size       => 10000000000,
        min_part_size_ratio => 0.01,
        method              => 'zstd',
      },
    },
  },
  profiles         => {
    production => {
      use_uncompressed_cache => 0,
      log_queries => 1,
      max_memory_usage => ceiling($facts['memory']['system']['total_bytes'] * 0.7),
    },
  },
  quotas           => {
    production => {
      interval => [
        {
          duration       => 3600,
          queries        => 200,
          erros          => 10,
          result_rows    => 0,
          read_rows      => 0,
          execution_time => 0,
        }
      ],
    },
  },
  users            => {
    alice => {
      password => '02472d6a1e23f73b37481bbd67949a5d16cbaf3d71770696f20a0cd773a2e682',
      quota    => 'production',
      profile  => 'production',
      network  => {
        ip => ['::/0'],
      },
    },
  },
  replication      => {
    zookeeper_servers => ['zookeeper1.local:2181', 'zookeeper2.local:2181', 'zookeeper3.local:2181'],
    macros => {
      cluster => 'Clickhouse_cluster',
      replica => $facts['networking']['fqdn'],
    },
  },
  remote_servers   => {
    replicated           => {
      shard => {
        internal_replication => true,
        replica              => ['host1.local:9000', 'host2.local:9000'],
      },
    },
    segmented            => {
      shard1 => {
        weight               => 1,
        internal_replication => true,
        replica              => ['host1.local:9000'],
      },
      shard2 => {
        weight               => 2,
        internal_replication => true,
        replica              => ['host2.local:9000'],
      },
    },
    segmented_replicated => {
      shard1 => {
        internal_replication => true,
        replica              => ['host1.local:9000', 'host2.local:9000'],
      },
      shard2 => {
        internal_replication => true,
        replica              => ['host3.local:9000', 'host4.local:9000'],
      },
    },
  },
}

Reference

Classes

Public Classes

Private Classes

  • clickhouse::client::install: Private class for managing Clickhouse client package.
  • clickhouse::params: Private class for setting default Clickhouse parameters.
  • clickhouse::server::config: Private class for Clickhouse server configuration.
  • clickhouse::server::install: Private class for managing Clickhouse server package.
  • clickhouse::server::resources: Private class for applying Clickhouse resources.
  • clickhouse::server::service: Private class for managing the Clickhouse service.

Defined types

Resource types

Functions

Limitations

For a list of supported operating systems, see metadata.json

Development

Please feel free to fork, modify, create issues, bug reports and pull requests.