Forge Home

ct2ls

Installs a daemon script to import logs from cloudtrail to logstash

10,364 downloads

9,281 latest version

4.5 quality score

We run a couple of automated
scans to help you access a
module's quality. Each module is
given a score based on how well
the author has formatted their
code and documentation and
modules are also checked for
malware using VirusTotal.

Please note, the information below
is for guidance only and neither of
these methods should be considered
an endorsement by Puppet.

Version information

  • 0.1.0 (latest)
  • 0.0.2
  • 0.0.1
released Mar 23rd 2015
This version is compatible with:
  • , ,

Start using this module

  • r10k or Code Manager
  • Bolt
  • Manual installation
  • Direct download

Add this module to your Puppetfile:

mod 'evenup-ct2ls', '0.1.0'
Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add evenup-ct2ls
Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install evenup-ct2ls --version 0.1.0

Direct download is not typically how you would use a Puppet module to manage your infrastructure, but you may want to download the module in order to inspect the code.

Download

Documentation

evenup/ct2ls — version 0.1.0 Mar 23rd 2015

Why do I want this?

You run some or all of your environment in Amazon Web Services, want to enable CloudTrail to maintain audit logs of what is going on, and want those logs to (almost) magically appear in logstash.

How does it work?

Amazon has added the ability to log all API calls via the CloudTrail service. When CloudTrail is enabled, it writes log files to a S3 bucket with the option of adding a notification via SNS when new logs are dropped. Adding a SQS receiver to the SNS topic allows you to poll for new log files that have been dropped.

A simple script included with this puppet module subscribes to the SQS queue, grabs the referenced log file from S3, formats the log messages for logstash and adds them to a redis list for consumption by logstash.

A sample AWS policy would look like this:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1400869009000",
      "Effect": "Allow",
      "Action": [
        "sqs:GetQueueUrl",
        "sqs:ReceiveMessage",
        "sqs:DeleteMessage"
      ],
      "Resource": [
        "arn:aws:sqs:<region>:<account_ID>:<queue_name>"
      ]
    }, {
      "Sid": "Stmt1400852117000",
      "Effect": "Allow",
      "Action": [
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::<bucket_name>/*"
      ]
    }
  ]
}

Usage

This script is conveniently wrapped in a puppet module to simplify installation.

Known Issues:

Only tested on CentOS 6

TODO:


[ ] Make evenup/ruby module optional

License:


Released under the Apache 2.0 licence

Contribute:

  • Fork it
  • Create a topic branch
  • Improve it with your awesome ideas or fix my dumb bugs
  • Push new topic branch
  • Submit a PR