cockroachdb

Table of Contents

1. Description
2. Setup - Getting started with cockroachdb
   • What cockroachdb affects
   • Setup requirements
   • Beginning with cockroachdb
3. Example Usage
4. Limitations and Known Issues
5. Development - Guide for contributing to the module

Description

This is a CockroachDB module for Puppet. It will help you set up a database cluster quickly and provides basic functionality for managing it using Bolt tasks. The module is created using PDK for Open Source Puppet and using the Open Source version of CockroachDB.

The module was initially created as part of a project in the subject Infrastructure as Code (IMT3005) at NTNU Gjøvik.

Setup

What cockroachdb affects

Dependencies installed by cockroachdb by default:

• tar
• wget
• puppet-archive v4.2.0
• puppetlabs-stdlib >= 4.13.1 < 7.0.0

Setup Requirements

At the moment the module requires that database nodes are identified by FQDNs, so DNS should be configured prior to use.

To use secure mode, you need to handle the certificates either manually, or using other modules/programs.

Beginning with cockroachdb

First add the module to your Puppetfile, see the Forge instructions. Then all you need in your manifest to get started is:

Insecure mode

class { 'cockroachdb':
  node1ip     => 'db0',
  node2ip     => 'db1',
  node3ip     => 'db2',
  secure_mode => false,
}

Secure mode

class { 'cockroachdb':
  node1ip     => 'db0',
  node2ip     => 'db1',
  node3ip     => 'db2',
  secure_mode => true,
  certs_dir   => '/root/certs',
}

Client To just install the cockroach client and not run it as a server, just add include cockroachdb::client to your Puppet manifest.

Usage

Specifying different cockroachdb version:

class { 'cockroachdb':
  node1ip        => 'db0',
  node2ip        => 'db1',
  node3ip        => 'db2',
  package_ensure => 'v19.1.0.linux-amd64',
  secure_mode    => false,
}

Adding additional parameters to cockroachdb that are not directly supported in the module:

class { 'cockroachdb':
  node1ip           => 'db0',
  node2ip           => 'db1',
  node3ip           => 'db2',
  secure_mode       => false,
  additional_params => '--store=/mnt/ssd01',
}

Initializing a cluster using bolt tasks:

To then initialize the database cluster, use:

bolt task run cockroachdb -n db0 insecure=true

Verify that the cluster is online:

bolt task run cockroachdb::node_ls -n db0 insecure=true

Limitations

Limitations:

• Only a three node initial cluster is supported. But still works with more nodes joining later
• Can not switch from secure to insecure mode

Known Issues:

Warnings:

Development

Contribution guidelines for this project

Changelog

Changelog

Reference

Table of Contents

Classes

• cockroachdb: Recives parameters from the user and initializes the cockroach::install and cockroach::config classes.
• cockroachdb::client: Install the cockroachdb client
• cockroachdb::config: This class is called from the cockroachdb class to configure the program.
• cockroachdb::install: This class is called from the the cockroachdb class for installation.
• cockroachdb::params: Default values for cockroachdb module.
• cockroachdb::service: This class is called from the cockroachdb class to configure the service.

Tasks

• init: Initialize a database cluster
• node_decommission: Decommission a node from the cluster
• node_ls: List nodes in a cluster.
• node_status: Check status of a node in the cluster.
• quit: Stop a node in a cluster.
• restart_service: Restart cockroachdb service
• sql: Run a sql command on a cluster

Classes

cockroachdb

Recives parameters from the user and initializes the cockroach::install and cockroach::config classes.

Parameters

The following parameters are available in the cockroachdb class.

servicepath

Data type: Optional[Stdlib::Absolutepath]

Location of where the ".service" file is to be saved. If the parameter is not defined it will be saved in "/etc/systemd/system".

Default value: $cockroachdb::params::servicepath

workingdirectory

Data type: Optional[Stdlib::Absolutepath]

Location of where the working directory will be created. If the parameter is not defined it will be created in "/var/lib/cockroach".

Default value: $cockroachdb::params::workingdirectory

dependencies

Data type: Optional[Tuple]

Consists of dependencies 'tar' and 'wget' by default. Failing to include these dependencies while defining this parameter yourself may result in dependency issues.

Default value: $cockroachdb::params::dependencies

description

Data type: Optional[String]

Cockroachdb node description.

Default value: $cockroachdb::params::description

cache

Data type: Optional[String]

The total size for caches, shared if there are multiple storage devices, noted with a decimal or '%'.

Default value: $cockroachdb::params::cache

maxsqlmemory

Data type: Optional[String]

The maximum in-memory storage capaciy available to store temprary data for SQL queries. Noted with a decimal point or '%'.

Default value: $cockroachdb::params::maxsqlmemory

timeoutstopsec

Data type: Optional[String]

Configures the time to wait for each ExecStop command for systemd.

Default value: $cockroachdb::params::timeoutstopsec

restart

Data type: Optional[String]

Defines when the service is to be restarted. Configured with always by default and will restart on every restart setting and exit cause.

Default value: $cockroachdb::params::restart

restartsec

Data type: Optional[String]

Defines the time to sleep before restarting the service.

Default value: $cockroachdb::params::restartsec

standardoutput

Data type: Optional[String]

Controls where the file descriptor 1 (STDOUT) of the executed processes is connected to.

Default value: $cockroachdb::params::standardoutput

standarderror

Data type: Optional[String]

Controls where the file descriptor 2 (STDERR) of the executed process is connected to.

Default value: $cockroachdb::params::standarderror

syslogidentifier

Data type: Optional[String]

Sets the process name ("syslog tag") to prefix log lines sent to the logging system or the kernel log buffer.

Default value: $cockroachdb::params::syslogidentifier

user

Data type: Optional[String]

Sets the user/group that the processes are executed as.

Default value: $cockroachdb::params::user

node1ip

Data type: String

Sets the ip-address for node one. Set to localhost when in single node mode.

Default value: $cockroachdb::params::node1ip

node2ip

Data type: Optional[String]

Sets the ip-address for node two. Skip for single node mode.

Default value: $cockroachdb::params::node2ip

node3ip

Data type: Optional[String]

Sets the ip-address for node three. Skip for single node mode.

Default value: $cockroachdb::params::node3ip

install_path

Data type: Optional[Stdlib::Absolutepath]

Location where the program binary file is to be located.

Default value: $cockroachdb::params::install_path

package_name

Data type: Optional[String]

Name of the package to install.

Default value: $cockroachdb::params::package_name

package_ensure

Data type: String

Version number and linux distribution name.

Default value: $cockroachdb::params::package_ensure

repository_url

Data type: Optional[String]

Base repository URL.

Default value: $cockroachdb::params::repository_url

archive_name

Data type: Optional[String]

Archive path of the binary file in the base repository.

Default value: $cockroachdb::params::archive_name

cockroachdb_package_source

Data type: Optional[String]

Full binary package source.

Default value: $cockroachdb::params::cockroachdb_package_source

additional_params

Data type: Optional[String]

Additional params and flags to pass to cockroachdb on startup. Must be provided as fully valid cockroachdb flags.

Default value: $cockroachdb::params::additional_params

secure_mode

Data type: Boolean

If cockroachdb secure mode is enabled or not. Currently only supports insecure mode.

Default value: $cockroachdb::params::secure_mode

locality

Data type: Optional[Array[String, 1]]

Array of Hashes containing locality information for nodes.

Default value: $cockroachdb::params::locality

certs_dir

Data type: Optional[Stdlib::Absolutepath]

The directory where CockroachDB can find certificates.

Default value: $cockroachdb::params::certs_dir

dns_mode

Data type: Optional[Enum['hostname', 'fqdn', 'ipaddress', 'custom']]

Use FQDN, hostnames, IPs or custom facts to identify nodes.

Default value: $cockroachdb::params::dns_mode

dns_mode_custom

Data type: Optional[String]

Custom fact used to identify nodes. Can be any valid facter fact, like custom facts.

Default value: $cockroachdb::params::dns_mode_custom

cockroachdb::client

Install the cockroachdb client

Examples

include cockroachdb::client

Parameters

The following parameters are available in the cockroachdb::client class.

workingdirectory

Data type: Optional[Stdlib::Absolutepath]

Location of where the working directory will be created. If the parameter is not defined it will be created in "/var/lib/cockroach".

Default value: $cockroachdb::params::workingdirectory

dependencies

Data type: Optional[Tuple]

Consists of dependencies 'tar' and 'wget' by default. Failing to include these dependencies while defining this parameter yourself may result in dependency issues.

Default value: $cockroachdb::params::dependencies

install_path

Data type: Optional[Stdlib::Absolutepath]

Location where the program binary file is to be located.

Default value: $cockroachdb::params::install_path

package_name

Data type: Optional[String]

Name of the package to install.

Default value: $cockroachdb::params::package_name

package_ensure

Data type: String

Version number and linux distribution name.

Default value: $cockroachdb::params::package_ensure

repository_url

Data type: Optional[String]

Base repository URL.

Default value: $cockroachdb::params::repository_url

archive_name

Data type: Optional[String]

Archive path of the binary file in the base repository.

Default value: $cockroachdb::params::archive_name

cockroachdb_package_source

Data type: Optional[String]

Full binary package source.

Default value: $cockroachdb::params::cockroachdb_package_source

cockroachdb::config

This class is called from the cockroachdb class to configure the program.

cockroachdb::install

This class is called from the the cockroachdb class for installation.

cockroachdb::params

Default values for cockroachdb module.

cockroachdb::service

This class is called from the cockroachdb class to configure the service.

Tasks

init

Initialize a database cluster

Supports noop? false

Parameters

host

Data type: Optional[String[1]]

The server host and port number to connect to. This can be the address of any node in the cluster.

port

Data type: Optional[Integer[1]]

The server port to connect to. Note: The port number can also be specified via --host.

user

Data type: Optional[String[1]]

The SQL user that will own the client session.

insecure

Data type: Optional[Boolean]

Use an insecure connection.

certs_dir

Data type: Optional[String[1]]

The path to the certificate directory containing the CA and client certificates and client key.

url

Data type: Optional[String[1]]

A connection URL to use instead of the other arguments.

node_decommission

Decommission a node from the cluster

Supports noop? false

Parameters

host

Data type: Optional[String[1]]

The server host and port number to connect to. This can be the address of any node in the cluster.

port

Data type: Optional[Integer[1]]

The server port to connect to. Note: The port number can also be specified via --host.

user

Data type: Optional[String[1]]

The SQL user that will own the client session.

insecure

Data type: Optional[Boolean]

Use an insecure connection.

certs_dir

Data type: Optional[String[1]]

The path to the certificate directory containing the CA and client certificates and client key.

url

Data type: Optional[String[1]]

A connection URL to use instead of the other arguments.

wait

Data type: Optional[Enum[all,none]]

When to return to the client. Possible values: all, none.

node_ls

List nodes in a cluster.

Supports noop? false

Parameters

host

Data type: Optional[String[1]]

The server host and port number to connect to. This can be the address of any node in the cluster.

port

Data type: Optional[Integer[1]]

The server port to connect to. Note: The port number can also be specified via --host.

user

Data type: Optional[String[1]]

The SQL user that will own the client session.

insecure

Data type: Optional[Boolean]

Use an insecure connection.

certs_dir

Data type: Optional[String[1]]

The path to the certificate directory containing the CA and client certificates and client key.

url

Data type: Optional[String[1]]

A connection URL to use instead of the other arguments.

format

Data type: Optional[Enum[tsv,csv,table,records,sql,html]]

How to display table rows printed to the standard output. Possible values: tsv, csv, table, records, sql, html.

timeout

Data type: Optional[Pattern[/^[0-9]{1,}(s|m|h)$/]]

Set the duration of time that the subcommand is allowed to run before it returns an error and prints partial information. The timeout is specified with a suffix of s for seconds, m for minutes, and h for hours. If this flag is not set, the subcommand may hang.

node_status

Check status of a node in the cluster.

Supports noop? false

Parameters

host

Data type: Optional[String[1]]

The server host and port number to connect to. This can be the address of any node in the cluster.

port

Data type: Optional[Integer[1]]

The server port to connect to. Note: The port number can also be specified via --host.

user

Data type: Optional[String[1]]

The SQL user that will own the client session.

insecure

Data type: Optional[Boolean]

Use an insecure connection.

certs_dir

Data type: Optional[String[1]]

The path to the certificate directory containing the CA and client certificates and client key.

url

Data type: Optional[String[1]]

A connection URL to use instead of the other arguments.

format

Data type: Optional[Enum[tsv,csv,table,records,sql,html]]

How to display table rows printed to the standard output. Possible values: tsv, csv, table, records, sql, html.

timeout

Data type: Optional[Pattern[/^[0-9]{1,}(s|m|h)$/]]

Set the duration of time that the subcommand is allowed to run before it returns an error and prints partial information. The timeout is specified with a suffix of s for seconds, m for minutes, and h for hours. If this flag is not set, the subcommand may hang.

all

Data type: Optional[Boolean]

Show all node details.

decommission

Data type: Optional[Boolean]

Show node decommissioning details.

ranges

Data type: Optional[Boolean]

Show node details for ranges and replicas.

stats

Data type: Optional[Boolean]

Show node disk usage details.

quit

Stop a node in a cluster.

Supports noop? false

Parameters

host

Data type: Optional[String[1]]

The server host and port number to connect to. This can be the address of any node in the cluster.

port

Data type: Optional[Integer[1]]

The server port to connect to. Note: The port number can also be specified via --host.

user

Data type: Optional[String[1]]

The SQL user that will own the client session.

insecure

Data type: Optional[Boolean]

Use an insecure connection.

certs_dir

Data type: Optional[String[1]]

The path to the certificate directory containing the CA and client certificates and client key.

url

Data type: Optional[String[1]]

A connection URL to use instead of the other arguments.

decommission

Data type: Optional[Boolean]

If specified, the node will be permanently removed instead of temporarily stopped.

restart_service

Restart cockroachdb service

Supports noop? false

sql

Run a sql command on a cluster

Supports noop? false

Parameters

host

Data type: Optional[String[1]]

The server host and port number to connect to. This can be the address of any node in the cluster.

port

Data type: Optional[Integer[1]]

The server port to connect to. Note: The port number can also be specified via --host.

user

Data type: Optional[String[1]]

The SQL user that will own the client session.

insecure

Data type: Optional[Boolean]

Use an insecure connection.

certs_dir

Data type: Optional[String[1]]

The path to the certificate directory containing the CA and client certificates and client key.

url

Data type: Optional[String[1]]

A connection URL to use instead of the other arguments.

database

Data type: Optional[String[1]]

A database name to use as current database in the newly created session.

echo_sql

Data type: Optional[String[1]]

Reveal the SQL statements sent implicitly by the command-line utility.

execute

Data type: String[1]

Execute SQL statements directly from the command line, without opening a shell. Non-optional, to prevent shell from opening

format

Data type: Optional[Enum[tsv,csv,table,raw,records,sql,html]]

How to display table rows printed to the standard output. Possible values: tsv, csv, table, raw, records, sql, html.

safe_updates

Data type: Optional[Boolean]

Disallow potentially unsafe SQL statements.

set

Data type: Optional[String[1]]

Set a client-side option before executing SQL statements from the command line via --execute.

Change log

All notable changes to this project will be documented in this file. The format is based on Keep a Changelog and this project adheres to Semantic Versioning.

v1.1.0 (2019-10-31)

Full Changelog

Merged pull requests:

• Acceptance testing and single node mode #99 (Hexoplon)
• set service enable equal to true #98 (Hexoplon)
• Fixed param name, install referenced wrong variable name #97 (Hexoplon)
• client class to install cockroachdb client #96 (Hexoplon)

v1.0.1 (2019-10-29)

Full Changelog

Merged pull requests:

• Hotfix service template, missing start command #94 (Hexoplon)

v1.0.0 (2019-10-29)

Full Changelog

Merged pull requests:

• Choose which fact facter should use to get advertise-address #92 (Hexoplon)
• Changed hardcoded names with already existing class params #89 (Hexoplon)
• Secure mode #52 #88 (Hexoplon)
• Added RestartPreventExitStatus to service #87 (Hexoplon)
• Task quit #85 (Hexoplon)

v0.3.0 (2019-10-23)

Full Changelog

Merged pull requests:

• Fix locality flag #83 (Hexoplon)
• Explicitly enable insecure mode #77 (Hexoplon)
• Sql bolt task #72 (Hexoplon)
• added additional_params #69 #70 (Hexoplon)
• hotfix executable #64 (martinaamodt)
• fixed comma with k #61 (joachiwi)
• hotfix init task type Boolean. v0.2.1 #58 (Hexoplon)
• Added Bolt init task. Various small merges #57 (martinaamodt)
• added bolt init task #55 (Hexoplon)
• Added stdlib::absolutepath type to relevant params #46 (Hexoplon)
• flag= --locality #43 (joachiwi)
• Params class #28 (Hexoplon)
• Service template #25 (martinaamodt)
• puppet-archive #14 (Hexoplon)
• Task node #73 (Hexoplon)
• Revamp testing #27 (Hexoplon)
• config test service #23 (joachiwi)
• correct version number #12 (martinaamodt)
• Service #10 (martinaamodt)
• Refactor config #9 (Hexoplon)

v0.1.5 (2019-10-11)

Full Changelog

Merged pull requests:

• Updated to pdk 1.14 #6 (Hexoplon)

v0.1.4 (2019-10-10)

Full Changelog

Merged pull requests:

• User and dir change #5 (joachiwi)

v0.1.3 (2019-10-10)

Merged pull requests:

• Create config #1 (martinaamodt)

* This Change Log was automatically generated by github_changelog_generator