concat

#Concat

####Table of Contents

1. Overview
2. Module Description - What the module does and why it is useful
3. Setup - The basics of getting started with concat
   • What concat affects
   • Setup requirements
   • Beginning with concat
4. Usage - Configuration options and additional functionality
   • API deprecations
5. Reference - An under-the-hood peek at what the module is doing and how
6. Limitations - OS compatibility, etc.
7. Development - Guide for contributing to the module

##Overview

This module constructs files from multiple fragments in an ordered way.

##Module Description

This module lets you use many concat::fragment{} resources throughout your modules to construct a single file at the end. It does this through a shell (or ruby) script and a temporary holding space for the fragments.

##Setup

###What concat affects

• Installs concatfragments.[sh|rb] based on platform.
• Adds a concat/ directory into Puppets vardir.

###Beginning with concat

To start using concat you need to create:

• A concat{} resource for the final file.
• One or more concat::fragment{}'s.

A minimal example might be:

concat { '/tmp/file':
  ensure => present,
}

concat::fragment { 'tmpfile':
  target  => '/tmp/file'
  content => 'test contents',
  order   => '01'
}

##Usage

Please be aware that there have been a number of API deprecations.

If you wanted a /etc/motd file that listed all the major modules on the machine. And that would be maintained automatically even if you just remove the include lines for other modules you could use code like below, a sample /etc/motd would be:

Local sysadmins can also append to the file by just editing /etc/motd.local their changes will be incorporated into the puppet managed motd.

class motd {
  $motd = '/etc/motd'

  concat { $motd:
    owner => 'root',
    group => 'root',
    mode  => '0644'
  }

  concat::fragment{ 'motd_header':
    target  => $motd,
    content => "\nPuppet modules on this server:\n\n",
    order   => '01'
  }

  # local users on the machine can append to motd by just creating
  # /etc/motd.local
  concat::fragment{ 'motd_local':
    target => $motd,
    source => '/etc/motd.local',
    order  => '15'
  }
}

# used by other modules to register themselves in the motd
define motd::register($content="", $order=10) {
  if $content == "" {
    $body = $name
  } else {
    $body = $content
  }

  concat::fragment{ "motd_fragment_$name":
    target  => '/etc/motd',
    order   => $order,
    content => "    -- $body\n"
  }
}

To use this you'd then do something like:

class apache {
  include apache::install, apache::config, apache::service

  motd::register{ 'Apache': }
}

##Reference

###Classes

####Public classes

####Private classes

• concat::setup: Sets up the concat script/directories.

###Parameters

###Defines

####concat

#####ensure Controls if the combined file is present or absent.

######Example

• ensure => present
• ensure => absent

#####path Controls the destination of the file to create.

######Example

• path => '/tmp/filename'

#####owner Set the owner of the combined file.

######Example

• owner => 'root'

#####group Set the group of the combined file.

######Example

• group => 'root'

#####mode Set the mode of the combined file.

######Example

• mode => '0644'

#####warn Determine if a warning message should be added at the top of the file to let users know it was autogenerated by Puppet.

######Example

• warn => true
• warn => false

#####warn_message Set the contents of the warning message.

######Example

• warn_message => 'This file is autogenerated!'

#####force Determine if empty files are allowed when no fragments were added.

######Example

• force => true
• force => false

#####backup Controls the filebucket behavior used for the file.

######Example

• backup => 'puppet'

#####replace Controls if Puppet should replace the destination file if it already exists.

######Example

• replace => true
• replace => false

#####order Controls the way in which the shell script chooses to sort the files. It's rare you'll need to adjust this.

######Allowed Values

• order => 'alpha'
• order => 'numeric'

#####ensure_newline Ensure there's a newline at the end of the fragments.

######Example

• ensure_newline => true
• ensure_newline => false

####concat::fragment

#####target Choose the destination file of the fragment.

######Example

• target => '/tmp/testfile'

#####content Create the content of the fragment.

######Example

• content => 'test file contents'

#####source Find the sources within Puppet of the fragment.

######Example

• source => 'puppet:///modules/test/testfile'
• source => ['puppet:///modules/test/1', 'puppet:///modules/test/2']

#####order Order the fragments.

######Example

• order => '01'

#####ensure Control the file of fragment created.

######Example

• ensure => 'present'
• ensure => 'absent'
• ensure => 'file'
• ensure => 'directory'

#####mode Set the mode of the fragment.

######Example

• mode => '0644'

#####owner Set the owner of the fragment.

######Example

• owner => 'root'

#####group Set the group of the fragment.

######Example

• group => 'root'

#####backup Control the filebucket behavior for the fragment.

######Example

• backup => 'puppet'

API deprecations

Since version 1.0.0

concat{} warn parameter

concat { '/tmp/file':
  ensure => present,
  warn   => 'true',  # generates stringified boolean value warning
}

Using stringified Boolean values as the warn parameter to concat is deprecated, generates a catalog compile time warning, and will be silently treated as the concatenated file header/warning message in a future release.

The following strings are considered a stringified Boolean value:

• 'true'
• 'yes'
• 'on'
• 'false'
• 'no'
• 'off'

Please migrate to using the Puppet DSL's native Boolean data type.

concat{} gnu parameter

concat { '/tmp/file':
  ensure => present,
  gnu    => $foo,    # generates deprecation warning
}

The gnu parameter to concat is deprecated, generates a catalog compile time warning, and has no effect. This parameter will be removed in a future release.

Note that this parameter was silently ignored in the 1.0.0 release.

concat::fragment{} ensure parameter

concat::fragment { 'cpuinfo':
  ensure => '/proc/cpuinfo', # generates deprecation warning
  target => '/tmp/file',
}

Passing a value other than 'present' or 'absent' as the ensure parameter to concat::fragment is deprecated and generates a catalog compile time warning. The warning will become a catalog compilation failure in a future release.

This type emulates the Puppet core file type's disfavored ensure semantics of treating a file path as a directive to create a symlink. This feature is problematic in several ways. It copies an API semantic of another type that is both frowned upon and not generally well known. It's behavior may be surprising in that the target concatenated file will not be a symlink nor is there any common file system that has a concept of a section of a plain file being symbolically linked to another file. Additionally, the behavior is generally inconsistent with most Puppet types in that a missing source file will be silently ignored.

If you want to use the content of a file as a fragment please use the source parameter.

concat::fragment{} mode/owner/group parameters

concat::fragment { 'foo':
  target  => '/tmp/file',
  content => 'foo',
  mode    => $mode,       # generates deprecation warning
  owner   => $owner,      # generates deprecation warning
  group   => $group,      # generates deprecation warning
}

The mode parameter to concat::fragment is deprecated, generates a catalog compile time warning, and has no effect.

The owner parameter to concat::fragment is deprecated, generates a catalog compile time warning, and has no effect.

The group parameter to concat::fragment is deprecated, generates a catalog compile time warning, and has no effect.

These parameters had no user visible effect in version 1.0.0 and will be removed in a future release.

concat::fragment{} backup parameter

concat::fragment { 'foo':
  target  => '/tmp/file',
  content => 'foo',
  backup  => 'bar',       # generates deprecation warning
}

The backup parameter to concat::fragment is deprecated, generates a catalog compile time warning, and has no effect. It will be removed in a future release.

In the 1.0.0 release this parameter controlled file bucketing of the file fragment. Bucketting the fragment(s) is redundant with bucketting the final concatenated file and this feature has been removed.

class { 'concat::setup': }

include concat::setup     # generates deprecation warning

class { 'concat::setup: } # generates deprecation warning

The concat::setup class is deprecated as a public API of this module and should no longer be directly included in the manifest. This class may be removed in a future release.

Parameter validation

While not an API depreciation, users should be aware that all public parameters in this module are now validated for at least variable type. This may cause validation errors in a manifest that was previously silently misbehaving.

##Limitations

This module has been tested on:

• RedHat Enterprise Linux (and Centos) 5/6
• Debian 6/7
• Ubuntu 12.04

Testing on other platforms has been light and cannot be guaranteed.

#Development

Puppet Labs modules on the Puppet Forge are open projects, and community contributions are essential for keeping them great. We can’t access the huge number of platforms and myriad of hardware, software, and deployment configurations that Puppet is intended to serve.

We want to keep it as easy as possible to contribute changes so that our modules work in your environment. There are a few guidelines that we need contributors to follow so that we can have a chance of keeping on top of things.

You can read the complete module contribution guide on the Puppet Labs wiki.

###Contributors

The list of contributors can be found at:

https://github.com/puppetlabs/puppetlabs-concat/graphs/contributors

2014-05-14 1.1.0

Summary

This release is primarily a bugfix release since 1.1.0-rc1.

Features:

• Improved testing, with tests moved to beaker

Bugfixes:

• No longer attempts to set fragment owner and mode on Windows
• Fix numeric sorting
• Fix incorrect quoting
• Fix newlines

2014-01-03 1.1.0-rc1

Summary:

This release of concat was 90% written by Joshua Hoblitt, and the module team would like to thank him for the huge amount of work he put into this release.

This module deprecates a bunch of old parameters and usage patterns, modernizes much of the manifest code, simplifies a whole bunch of logic and makes improvements to almost all parts of the module.

The other major feature is windows support, courtesy of luisfdez, with an alternative version of the concat bash script in ruby. We've attempted to ensure that there are no backwards incompatible changes, all users of 1.0.0 should be able to use 1.1.0 without any failures, but you may find deprecation warnings and we'll be aggressively moving for a 2.0 to remove those too.

For further information on deprecations, please read: https://github.com/puppetlabs/puppetlabs-concat/blob/master/README.md#api-deprecations

Removed:

• Puppet 0.24 support.
• Filebucket backup of all file resources except the target concatenated file.
• Default owner/user/group values.
• Purging of long unused /usr/local/bin/concatfragments.sh

Features:

• Windows support via a ruby version of the concat bash script.
• Huge amount of acceptance testing work added.
• Documentation (README) completely rewritten.
• New parameters in concat:
• ensure: Controls if the file should be present/absent at all.
• Remove requirement to include concat::setup in manifests.
• Made gnu parameter deprecated.
• Added parameter validation.

Bugfixes:

• Ensure concat::setup runs before concat::fragment in all cases.
• Pluginsync references updated for modern Puppet.
• Fix incorrect group parameter.
• Use $owner instead of $id to avoid confusion with $::id
• Compatibility fixes for Puppet 2.7/ruby 1.8.7
• Use LC_ALL=C instead of LANG=C
• Always exec the concatfragments script as root when running as root.
• Syntax and other cleanup changes.

2013-08-09 1.0.0

Summary:

Many new features and bugfixes in this release, and if you're a heavy concat user you should test carefully before upgrading. The features should all be backwards compatible but only light testing has been done from our side before this release.

Features:

• New parameters in concat:
• replace: specify if concat should replace existing files.
• ensure_newline: controls if fragments should contain a newline at the end.
• Improved README documentation.
• Add rspec:system tests (rake spec:system to test concat)

Bugfixes

• Gracefully handle \n in a fragment resource name.
• Adding more helpful message for 'pluginsync = true'
• Allow passing source and content directly to file resource, rather than defining resource defaults.
• Added -r flag to read so that filenames with \ will be read correctly.
• sort always uses LANG=C.
• Allow WARNMSG to contain/start with '#'.
• Replace while-read pattern with for-do in order to support Solaris.

CHANGELOG:

• 2010/02/19 - initial release
• 2010/03/12 - add support for 0.24.8 and newer

         - make the location of sort configurable
         - add the ability to add shell comment based warnings to
           top of files
         - add the ablity to create empty files
  

• 2010/04/05 - fix parsing of WARN and change code style to match rest of the code

         - Better and safer boolean handling for warn and force
         - Don't use hard coded paths in the shell script, set PATH
           top of the script
         - Use file{} to copy the result and make all fragments owned
           by root.  This means we can chnage the ownership/group of the
           resulting file at any time.
         - You can specify ensure => "/some/other/file" in concat::fragment
           to include the contents of a symlink into the final file.
  

• 2010/04/16 - Add more cleaning of the fragment name - removing / from the $name
• 2010/05/22 - Improve documentation and show the use of ensure =>
• 2010/07/14 - Add support for setting the filebucket behavior of files
• 2010/10/04 - Make the warning message configurable
• 2010/12/03 - Add flags to make concat work better on Solaris - thanks Jonathan Boyett
• 2011/02/03 - Make the shell script more portable and add a config option for root group
• 2011/06/21 - Make base dir root readable only for security
• 2011/06/23 - Set base directory using a fact instead of hardcoding it
• 2011/06/23 - Support operating as non privileged user
• 2011/06/23 - Support dash instead of bash or sh
• 2011/07/11 - Better solaris support
• 2011/12/05 - Use fully qualified variables
• 2011/12/13 - Improve Nexenta support
• 2012/04/11 - Do not use any GNU specific extensions in the shell script
• 2012/03/24 - Comply to community style guides
• 2012/05/23 - Better errors when basedir isnt set
• 2012/05/31 - Add spec tests
• 2012/07/11 - Include concat::setup in concat improving UX
• 2012/08/14 - Puppet Lint improvements
• 2012/08/30 - The target path can be different from the $name
• 2012/08/30 - More Puppet Lint cleanup
• 2012/09/04 - RELEASE 0.2.0
• 2012/12/12 - Added (file) $replace parameter to concat