Forge Home

Modules

Resources

PuppetEcosystemPuppet CommunityUpdates

confluent

This puppet module is used to install and configure the Confluent platform.
Project URLRSS FeedReport issues

27,658 downloads

3,481 latest version

4.5 quality score

We run a couple of automated
scans to help you access a
module's quality. Each module is
given a score based on how well
the author has formatted their
code and documentation and
modules are also checked for
malware using VirusTotal.

Please note, the information below
is for guidance only and neither of
these methods should be considered
an endorsement by Puppet.

Version information

  • 4.0.55 (latest)
  • 4.0.54
  • 4.0.53
  • 4.0.52
  • 4.0.51
  • 4.0.49
  • 4.0.48
  • 4.0.47
  • 4.0.46
  • 3.3.45
  • 3.3.44
  • 3.3.43
  • 3.3.42
  • 3.3.40
  • 3.3.39
  • 3.3.38
  • 3.3.35
  • 3.3.34
  • 3.3.33
  • 3.3.32
  • 3.3.31
  • 3.3.30
  • 3.3.29
  • 3.3.28
  • 3.3.27
  • 3.2.26
  • 3.2.25
  • 3.2.24
  • 3.2.23
  • 3.2.22
  • 3.2.21
released Sep 29th 2019
This version is compatible with:
  • ,

Start using this module

  • r10k or Code Manager
  • Bolt
  • Manual installation
  • Direct download

Add this module to your Puppetfile:

mod 'jcustenborder-confluent', '4.0.55'
Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add jcustenborder-confluent
Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install jcustenborder-confluent --version 4.0.55

Direct download is not typically how you would use a Puppet module to manage your infrastructure, but you may want to download the module in order to inspect the code.

Download

Documentation

jcustenborder/confluent — version 4.0.55 Sep 29th 2019

Introduction

This puppet module is used to install and configure the Confluent Platform. The documentation is available here.

Note on versioning

Puppet only allows version numbers in <major>.<minor>.<version> format. Due to this limitation the versioning schema matches the major and minor versions of the Confluent Platform. Meaning 3.2.1234 of the puppet module is for 3.2.x of the Confluent platform.

Usage

Zookeeper

Class installation

class{'confluent::zookeeper':
  zookeeper_id => '1',
  heap_size => '4000M'
}

Hiera Installation

include ::confluent::zookeeper

confluent::zookeeper::zookeeper_id: '1'
confluent::zookeeper::config:
  server.1:
    value: 'zookeeper-01.example.com:2888:3888'
  server.2:
    value: 'zookeeper-02.example.com:2888:3888'
  server.3:
    value: 'zookeeper-03.example.com:2888:3888'
confluent::zookeeper::heap_size: '4000M'

Kafka Broker

Class installation

class{'confluent::kafka::broker':
  broker_id => '1',
  zookeeper_connect => [
    'zookeeper-01:2181',
    'zookeeper-02:2181',
    'zookeeper-03:2181'
  ],
  heap_size => '4000M'
}

Heira installation

include ::confluent::kafka::broker

confluent::kafka::broker::broker_id: '1'
confluent::kafka::broker::heap_size: '4000M'
confluent::kafka::broker::zookeeper_connect:
  - 'zookeeper-01:2181',
  - 'zookeeper-02:2181',
  - 'zookeeper-03:2181'
confluent::kafka::broker::data_path: '/var/lib/kafka'

Kafka Connect

Distributed

Class Installation

class{'confluent::kafka::connect::distributed':
  heap_size => '4000M',
  bootstrap_servers => [
    'broker-01:9092',
    'broker-02:9092',
    'broker-03:9092'
  ],
  config => {
    'key.converter' => {
      'value' => 'io.confluent.connect.avro.AvroConverter'
    },
    'value.converter' => {
      'value' => 'io.confluent.connect.avro.AvroConverter'
    },
    'key.converter.schema.registry.url' => {
      'value' => 'http://schema-registry-01:8081'
    },
    'value.converter.schema.registry.url' => {
      'value' => 'http://schema-registry-01:8081'
    },
  },
}

Heira installation

include ::confluent::kafka::connect::distributed

 confluent::kafka::connect::distributed::heap_size: '4000M'
 confluent::kafka::connect::distributed::bootstrap_servers:
  - broker-01:9092
  - broker-02:9092
  - broker-03:9092
 confluent::kafka::connect::distributed::config:
   'key.converter':
     value: 'io.confluent.connect.avro.AvroConverter'
   'value.converter':
     value: 'io.confluent.connect.avro.AvroConverter'
   'key.converter.schema.registry.url':
     value: 'http://schema-registry-01.example.com:8081'
   'value.converter.schema.registry.url':
     value: 'http://schema-registry-01.example.com:8081'

Standalone

Class Installation

class{'confluent::kafka::connect::standalone':
  heap_size => '4000M',
  bootstrap_servers => [
    'broker-01:9092',
    'broker-02:9092',
    'broker-03:9092'
  ],
  config => {
    'key.converter' => {
      'value' => 'io.confluent.connect.avro.AvroConverter'
    },
    'value.converter' => {
      'value' => 'io.confluent.connect.avro.AvroConverter'
    },
    'key.converter.schema.registry.url' => {
      'value' => 'http://schema-registry-01:8081'
    },
    'value.converter.schema.registry.url' => {
      'value' => 'http://schema-registry-01:8081'
    },
  },
}

Heira installation

include ::confluent::kafka::connect::standalone

 confluent::kafka::connect::standalone::heap_size: '4000M'
 confluent::kafka::connect::standalone::bootstrap_servers:
  - broker-01:9092
  - broker-02:9092
  - broker-03:9092
 confluent::kafka::connect::standalone::config:
   'key.converter':
     value: 'io.confluent.connect.avro.AvroConverter'
   'value.converter':
     value: 'io.confluent.connect.avro.AvroConverter'
   'key.converter.schema.registry.url':
     value: 'http://schema-registry-01.example.com:8081'
   'value.converter.schema.registry.url':
     value: 'http://schema-registry-01.example.com:8081'

Schema Registry

Class installation

class {'confluent::schema::registry':
  heap_size => '1024M',
  kafkastore_connection_url => [
      'zookeeper-01:2181',
      'zookeeper-02:2181',
      'zookeeper-03:2181'
  ]
}

Hiera installation

include ::confluent::schema::registry

confluent::schema::registry::heap_size: '1024M'
confluent::schema::registry::kafkastore_connection_url:
  - 'zookeeper-01:2181'
  - 'zookeeper-02:2181'
  - 'zookeeper-03:2181'

Confluent Control Center

Class installation

class {'confluent::control::center':
  heap_size => '6g',
  zookeeper_connect => [
    'zookeeper-01:2181',
    'zookeeper-02:2181',
    'zookeeper-03:2181'
  ],
  bootstrap_servers => [
    'broker-01:9092',
    'broker-02:9092',
    'broker-03:9092'
  ],
  connect_cluster => [
    'kafka-connect-01.example.com:8083',
    'kafka-connect-02.example.com:8083',
    'kafka-connect-03.example.com:8083'
  ]
}

Hiera installation

include ::confluent::control::center

confluent::control::center::heap_size: '6g'
confluent::control::center::zookeeper_connect: 
  - 'zookeeper-01:2181'
  - 'zookeeper-02:2181'
  - 'zookeeper-03:2181'
confluent::control::center::bootstrap_servers: 
  - 'broker-01:9092'
  - 'broker-02:9092'
  - 'broker-03:9092'
confluent::control::center::connect_cluster: 
  - 'kafka-connect-01.example.com:8083'
  - 'kafka-connect-02.example.com:8083'
  - 'kafka-connect-03.example.com:8083'

Run tests

rake spec

Rebuild github pages

rake strings:gh_pages:update