The State of Open Source System Automation

The days of DIY system administration are rapidly coming to a close. Why? Because the open source tools available are just too good not to use. Presenting Bcfg2, Cfengine, Chef and Puppet.

Cfengine

Mark Burgess, the author of Cfengine, started his presentation byre-focusing the projector. The image was not blurry to start,just not completely in focus and I had no trouble reading the prior presenter’s slides; but it was very crisp after the adjustment! Such attention to detail inspired my confidence.

Cfengine is the granddaddy of open source configuration management tools, dating back to 1993 and Mark worked as a part-time sysadmin at the University of Oslo struggling with handling many different kinds of Unix and Unix-like systems.

Mark describes Cfengine as an agent-based change management system with “convergent” or “self-healing” behavior. (Cfengine will continuously return a system to the configured state or keep it there if it’s already there. Another way to put it, regardless of where you start from, you can always get to the defined state.)

The Cfengine language is a largely declarative language for describing desired or “promised” states. Like Bcfg2, the language is a pragmatic mix of declarative and procedural.

Cfengine includes a self-learning monitoring framework (to deal with an unknown environment) and a knowledge management framework (to help handle complexity of system configuration). Cfengine introduced the idea of”classes”, which are patterns in space and time and implicit if/then tests.

Examples of Cfengine Classes

  • The name of an operating system (Solaris or Red Hat Enterprise Linux)
  • Architecture (x86 or SPARC)
  • Time (Sunday, or 3 AM – 3:59 AM)
  • The name of a host, or a user-defined name of a group of hosts.
  • Any arbitrary string.

You can use Boolean logic with classes to select systems for a configuration promise. For example: Linux servers on x86 platform with -dev in the name should have their OS updated on Sunday at 3 AM.

Cfengine is model-based in the sense that you describe the model of the end state that you want.

Cfengine is self-documenting because you are using a declarative language.

Cfengine is lightweight (1.9 MB footprint). It has very few prerequisites (Berkeley DB library, crypto library and optional PCRE library). Today, Cfengine runs on everything from unmanned underwater vehicle to Nokia handheld phones to supercomputing clusters.

Because it’s a C binary with very few prerequisites, it has the largest span of systems it runs on out of all the open source CM tools.

At first, Cfengine was modeled as a computer immune system, helping a system stay healthy in an uncertain, changing, and possibly hostile environment.

The current philosophy of Cfengine is “promise theory”, where the defined state is promised by different system components(such as files, packages, processes, etc.), and Cfengine isa “promise engine” — an engine for keeping promises.

Key Principles of Cfengine’s Design:

  • Voluntary cooperation, local autonomy. Cfengine allows local control of policy in anticipation of consensus building amongst human administrators.Voluntary cooperation is expected; so Cfengine always pulls policy, never pushes it. A policy push is indistinguishable from attack. (Cfengine has had 3 security vulnerabilities in 17 years due to this principle.)
  • Pragmatism. Work with what you’ve got: allow shell commands.
  • Resilience Expect the unpredictable (therefore convergence back to promised state). Design allows for a single point of control without a single point of failure. (If a policy server goes away, the Cfengine agents on nodes will keep running using cached policy.)
  • Allow freedom. For example, allow use of package systems.Cfengine is about “constraint”, not “control”. The philosophy back of this is, “You do not control environments, you participate in environments.”
  • Convergence. Run Cfengine many times and the system should always get better and it should never worse. Always move closer to the promised state; or stay there. Stay there by always trying to move closer to it. (This counteracts the natural force of entropy which would result in system state drift over time.)

Promise Theory

Promise Theory is based on the key principles of convergence and autonomy.

Everything is a promise in cfengine language. Files promise to be there (and are created or copied by Cfengine if they are not); packages promise to be installed; processes promise to be running.

Cfengine configuration is composed of promises and patterns. A class is an example of a pattern; a list of packages to be installed is another (see example below).

Another practical pattern in Cfengine is abstraction of promise details so you can see at a glance what is promised, and can still drill down if necessary to get the promise details.

For example:

Abstracted Promise

copy_from => my_secure_cp("myfile","myserver")

Promise Body (Like “Contract Body” – Contains Details)

body copy_from my_secure_cp(file,server)
{
source      => "$(file)";
servers     => { "$(server)" };
compare     => "digest";
encrypt     => "true";
verify      => "true";
force_ipv4  => "false";
collapse_destination_dir => "false";
copy_size => irange("0","50000");
findertype => "MacOSX";
# etc etc
}

How Does Cfengine Work?

  1. The agent wakes up and classifies its environment (time, network address, OS, group defined by LDAP, etc.) This sets up all the classes.
  2. The agent reviews and execute promises. It may download the latest promise policy from a server; or use its local copy. Executing the promises, Cfengine will make 3 passes, checking everything and fulfilling as many promises as it can. For example, if SNMP packages promises to be installed, and SNMP daemon promises to be running; on the first pass, Cfengine could install SNMP package; on the second pass, it would start the daemon.
  3. The agent reports on success.

Cfengine Promise: Install the Postfix Package:

packages:
  "postfix"
     package_policy => "add",
     package_method => yum;

Cfengine Promise: Install Multiple Packages.

First, create a variable of type “list of strings” named @match_package.

Second, use an implicit loop over each element of the list (like in perl,@var is an array, $var is a scalar/string), and promise that package is added using YUM.

Loops are implicit in Cfengine, this is a powerful abstraction.

 vars:

  "match_package" slist => {
                           "apache2",
                           "apache2-mod_php5",
                           "php5"
                           };
 packages:

    "$(match_package)"

         package_policy => "add",
         package_method => yum;

Next: Chef

Comments on "The State of Open Source System Automation"

redmumba

We run Bcfg2 pretty extensively at our offices, and it certainly has its pluses and minuses. However, one of the things that is a real stick in the side is TGenshi, the Bcfg2 templating system. One of the great things about TGenshi is, well, it allows you to add logic to your file–so you can generate files from the Properties plugin, dynamically encrypt passwords, etc.. Great feature, right?

Debugging is AWFUL. The errors TGenshi throws by defaulty largely generic; for example, if you have a 100 line Python file being run in the template, and an error occurs anywhere, you’ll just get a message saying “Could not generate this file.” No line number, no raising of the original Python exception, nothing. If you want to do any serious work, you’ll have to write your own wrapper to catch errors–or at least a line number for what failed.

Bcfg2′s strongest feature is keeping everything the same on every server, so I would consider combining this for day-to-day maintenance, and maybe Puppet or cfengine for deployment.

Andrew

Reply
jblaine

We’ve been using cfengine 1+2 for 11 years.

Reply
vicente

we use cfengine2 with some logic of our own to control around 130 computers, and is very nice and powerfull, when you get to understand it.

Now we are thinking to get in puppet. I’d like to post soon to tell you how it was.

Reply

    We just ran into the sccm cenlit performance issue. We had a script blow out the sccm cenlit to the domain, and the support team did this on sunday at about 1am. Every sunday at 1am since then our VMware farm grinds to a halt. Cpu spikes, storage spikes.We found that sccm was launching a dir /s inventory on all of its cenlits on the 7 day aniversary. About 200 vms.Still not sure how we will fix this, but ideas would be appreciated.

    Reply

Definitely would love to start a website like yours. Wish I had the time. My site is so amateurish compared to yours, feel free to check it out: http://tinyurl.com/o55af8p Alex :)

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>