Intro to Automating System Administration with Cfengine 3

Here we see that the Perl Compatible Regular Expression (PCRE) library is being used. Success.

At this point you could do a make install, as root of course, and the binaries and support files will be installed into the default prefix of /usr/local unless you’ve change it. There is however another option.

Instead of using ‘make install’ you could manually install the files and have Cfengine keep them up to date for you. For example we often manually install the binaries into /var/cfengine/bin and the man pages into standard /usr/share/ man. We also keep copies of these files in our masterfiles repository. Then we configure promises in Cfengine to ensure that the files on client hosts match the files in the masterfiles repository. Using this method we are able to build new binaries once, copy them to masterfiles and Cfengine will automatically distribute them to all of our client hosts.

The basic grammar of Cfengine 3 looks like this:

type:
	classes::

	 "promiser" -> { "promisee1", "promisee2", ... }

		attribute_1 => value_1,
		attribute_2 => value_2,
		...
		attribute_n => value_n;

A class is a built-in if/then test, and if it is specified, the rule will hold only if that class is true for that machine. Examples of built-in classes can be: “Linux”, “Solaris”, “CentOS”, “Thursday”, “2:00am-3:00am”, a particular IP address on the host, etc. Examples of user-defined classes can be “database server” or all machines administered by our department.

Example: Hello World. 

commands:

        "/bin/echo hello world";

The promise type is “commands”, the promiser is the command string.

List of promise types

• commands – Run external commands
• files – Handle files (permissions, copying, etc.)
• edit_line – Handle files (content)
• interfaces – Network configuration
• methods – Methods are compound promises that refer to whole bundles of promises.
• packages – Package management
• processes – Process management
• storage – Disk and filesystem management

Another simple example: 

files:

"/tmp/test_plain" -> "John Smith",

	comment => "Make sure John's /tmp/test_plain exists",
	create  => "true";

Here we have the promisee on the right of the -> sign.

The promisee is “the abstract object to whom the promise is made”. This is for documentation. The commercial version of cfengine uses promisees to generate automated knowledge maps. The object can be the handle of another promise, recognizing an interest in the outcome, or an affected person who you might want to contact in case of emergency.

Example of a handle (id tag): 

files:

   "/tmp/test_plain" -> "John Smith",

		handle => "file_check",
		comment => "Make sure John's /tmp/test_plain exists",
		create  => "true";

Documenting a dependency with a handle: 

files:

  "/tmp/testcopy"

    depends_on    => { "file_check" },
    copy_from     => mycopy("/tmp/test_plain");

These are contrived examples; they document John Smith as somehow involved in this activity and show how handles can be used to document dependencies.

The above are not working examples. But this is, and you can run it to get familiar with Cfengine 3:

Make sure /tmp/test_plain exists. 

########################################################
body common control

{
version => "1.0";
bundlesequence  => { "test1"  };
}

########################################################

bundle agent test1

{

    files:

       "/tmp/test_plain" -> "John Smith",

            comment => "Make sure /tmp/test_plain exists",
            create  => "true";

}

First is the mandatory control section, where you can set certain values, and specify which promise bundles to run.

The next section is a promise bundle, which means a group of one or more promises. The bundle type is “agent” which means it will result in action on the part of Cfengine, or will be picked up by the Cfengine agent when it runs (cf-agent).

Each promise bundle has a name, and that name is referenced in bundlesequence in control.

Thus the above promise bundle is a promise that /tmp/test_plain exists; and specifies John Smith as an involved party.

Put the above in test.cf and run it with

cf-agent -f test.cf

If the file does not exist, Cfengine will create it.

Promise that ntpd and portmap are running. 

body common control

{
version => "1.0";
bundlesequence  => { "check_service_running"  };
}

bundle agent check_service_running

{
    vars:
        "service" slist => {"ntpd", "portmap"};
        "daemon_path" string => "/etc/init.d";

    processes:

            "${service}"
                comment => "Check processes running for '${service}'",
                restart_class => "restart_$(service)";

    commands:

        "${daemon_path}/${service} start"
            comment => "Execute the start command for the service",
            ifvarclass => "restart_${service}";

}

This uses variables (lists and strings); and when cf-agent hits the ${service} variable, which is a list, it loops over the list. This implicit looping is part of the power of Cfengine.

Promise content of /etc/resolv.conf. 

body common control
{
    version => "1.0";
    bundlesequence  => { "checkresolver" };
}

bundle agent checkresolver
{
    vars:

          "resolvers" slist => { "128.39.89.10", "158.36.85.10", "129.241.1.99" };

    files:

          "${sys.resolv}"

               edit_line => resolvconf("iu.hio.no cfengine.com",@{checkresolver.resolvers});
}

bundle edit_line resolvconf(search,list)
{
    delete_lines:

        "search.*";

    insert_lines:

         "search ${search}";
         "${list}";
}

“body common control{}” is the control promise body, which affects the operational behavior of Cfengine. In it, we define a version string, used in errors and reports.

Next we have a promise bundle “checkresolver{}” which uses:

• An “slist”, a list of scalar strings,
• “${sys.resolv}”, a system variable (built in to Cfengine) that containes the path to our resolv.conf file.
• checkresolver{} calls the promise bundle “resolvconf{}” which is of the editline promise type. Cfengine has built-in functionality for editing files, as this is a common system administration task, and editlines promises handle that. For example:
  • “delete_lines promise” will make sure we don’t end up with duplicate “search” lines which would be invalid for /etc/resolv.conf.
  • The “insert_lines” promise will make sure the file contains the specified data, which are a string (the search path) and an array (list of resolvers).

Note: we have to reference the @resolvers array using its full name, @checkresolver.resolvers, otherwise resolvconf will fail to find a @resolvers array within its own scope. The @resolvers array is in the scope of “checkresolver{}”.

You may have noticed the above Cfengine configuration is in two kinds of parts: a body or a bundle. What are they?

Bundle: A promise bundle is a collection of promises.

Body: A promise body is the part of a promise which details and constrains its nature. The body of a promise explains what it is about. Think of the body of a contract, or the body of a document. Cfengine “body” declarations divide up these details into standardized, paramaterizable, library units, so we can just write:

Snippet example of cf-agent client copying a file from cf-serverd (not a full working example). 

body copy_from my_secure_cp(from,server)
{
source      => "$(from)";
servers     => { "$(server)" };
compare     => "digest";
encrypt     => "true";
verify      => "true";
force_ipv4  => "false";
collapse_destination_dir => "false";
copy_size => irange("0","50000");
findertype => "MacOSX";
# etc etc
}

Think of the declarations in the promise as:

(cfengine-word) ⇒ (user-data-pattern)
body cfengine-word user-data-pattern
{
details
}

Working example of copying a file with cfengine, call it test_copy.cf. 

body common control
{
bundlesequence ⇒ { "testcopy" };
version ⇒ "1.2.3";
inputs ⇒ { "library.cf" };
}

bundle agent testcopy
{
files:
"/home/aleksey/testcopy1"
copy_from ⇒
my_copy_body_with_options("/home/aleksey/testcopy2","192.168.1.10");
}
body copy_from my_copy_body_with_options(sourcefile,sourceserver)
{
source ⇒ "$(sourcefile)";
servers ⇒ { "$(sourceserver)" };
copy_backup ⇒ "true";
purge ⇒ "true";
trustkey        ⇒ "true";
compare     ⇒ "digest";
encrypt     ⇒ "true";
verify      ⇒ "true";
force_ipv4  ⇒ "false";
collapse_destination_dir ⇒ "false";
copy_size ⇒ irange("0","50000");
# etc. etc.
}

Note: In order to get this example to work, we had to:

Modify “runagent control{}” and “server control{}” in promises.cf to remove mention of localhost (127.0.0.1 and ::1), replacing them with the IP address of the primary interface of 192.168.1.10.

Modify “server control{}” in promises.cf to add a non-root user to “allowusers” because we were running cf-serverd as a non-root user.

Here is the part of promises.cf we modified: 

body runagent control
{
hosts => {
          "192.168.1.10"
          # , "myhost.example.com:5308", ...
         };

}

#######################################################

body server control

{
allowconnects         => { "192.168.1.10" };
allowallconnects      => { "192.168.1.10" };
trustkeysfrom         => { "192.168.1.10" };

# make updates and runs happen in one

cfruncommand          => "$(sys.workdir)/bin/cf-agent -f failsafe.cf &&
$(sys.workdir)/bin/cf-agent";
allowusers            => { "root" , "aleksey" };
}

We also had to modify “server access_rules{}” in site.cf to set up the ACL to allow the file transfer. Running cf-serverd in verbose mode with debugging set to 1 will show more information about how the server handles requests.

Here are our changes to site.cf: . 

#######################################################
# server configuration
#######################################################

bundle server access_rules()
{
access:

  "/home/aleksey/testcopy2"

    admit   => { "192.168.1.10" };

roles:

  ".*"  authorize => { "aleksey" };
}

Generate a keypair for our cf-serverd:

cf-key

Start cf-serverd in verbose (non-forked) mode, debug level 1:

cf-serverd -v -d 1

Run our example code to copy the file from cf-serverd:

cf-agent -v test_copy.cf -k

More Information

cfengine contains far more features that we can touch upon in just this one introduction. To learn more about cfengine and how you can put it to use, check out the following resources:

• Cfengine 3 tutorial.
• Go through the cfengine 3 reference manual and try some of the promises in the cfengine 3 manual to see what cfengine 3 can do.
• Neil Watson’s cfengine 3 tutorial

Learning More

There will be a full day of Cfengine training at the LISA system administration conference in Boston in December 2011 http://www.usenix.org/event/lisa11/