Caos NSA and Perceus: All-in-one Cluster Software Stack

Silence the struggle around cluster software stack configuration. Caos NSA is a distribution that focuses on making things simple, easy to install and upgrade, and easy to manage.

Sometimes it’s a struggle to configure a cluster software stack. Usually the stack is split into a distribution (OS) and the cluster management tool. While there are some advantages to this separation, sometimes people want to have a cluster software stack that is all-in-one. There are several packages that satisfy this definition such as ROCKS and Clustermatic, but there is one that is arguably the best combination – Caos NSA and Perceus.

The all-in-one packages tend to build on top of existing distributions and then add a Cluster management System (CMS) and other tools to create the all-in-one package. One of the differentiators between these packages is the integration between the base distribution and the CMS. More integration requires more work but it does get the total package closer to the proverbial “cluster in a box.” More importantly it can make the configuration, deployment, updating, and monitoring of the cluster much easier.

But many of these base distributions are getting larger and heavier with every release, even for the core components. Typically, clusters don’t need this bloat (who needs to have 4 different music players on their nodes?) and just need a simple yet robust OS distribution. Caos NSA is a distribution that focuses on making things simple, easy to install and upgrade, and easy to manage.

Caos NSA

The Caos developers came together to form the Caos Foundation back in 2002. with Greg Kurtzer as the leader and founder. The Caos Foundation went public in 2003 with the goal of putting together an RPM based distribution of Linux for the community by the community. Originally they started both Centos and Caos. Centos was created as the build and development platform for Caos. Later the Centos project was split out on its own and the Caos team focused on the development of Caos-Linux which had different goals than Centos.

Caos 1 was developed and was used as an example of how one can manage an operating system at scale. It was a very usable Linux distribution but was somewhat close to Redhat or Centos. However, it did prove very useful in creating a new community supported distribution and it introduced some new ideas about how a distribution is built, maintained, and installed.

The Caos team took what it learned from Caos 1 and created Caos 2. Caos 2 is a general purpose distribution containing with about 1,800 packages. It was targeted at the advanced user and had a certain Debian’esq lightweight feel and efficiency to it.

Around the time that Caos 2 was released the development team realized that the world of Linux distributions was becoming more varied. In particular there were distributions that focused on the desktop, such as Ubuntu and Fedora Core, distributions that focused on servers, distributions that varied quickly and distributions that varied very slowly. In many ways distributions had become somewhat niche in that they focused on a particular “market.” More over, many distributions were becoming bloated and too “heavy” even at the core. Linux has always naturally excelled at being the lightest, simplest, fastest, and depending on perspective, secure choice. But now these distributions, were requiring new hardware, and giant overheads closer to Windows requirements. Obviously, not all Linux users require or much less want that, particularly for clusters. So Caos NSA Linux was developed to focus on where Linux has always naturally excelled – Simple, Lightweight, fast, easy, secure… NSA: Node, Server and Appliances. Caos NSA is focused on making a stable, somewhat slowly varying Linux distribution that can serve, not only as a great base server-like OS, but as a cluster distribution.

If you ask cluster people the attributes they want in a distribution, you are likely to get a laundry list of requirements. But, it’s fairly easy to get some consensus on some things. In no particular order these things are typically,

Requirements for Cluster distribution

  1. Stable
  2. Easy to install
  3. Easy to upgrade
  4. Robust
  5. Light weight with long term support

So Caos team focused on these things and developed a very stable, uncomplicated, easy to install and upgrade, distribution they call Caos NSA. They also included a new admin tool called Sidekick that allows you to administer the installed distribution. They use the Caos 2 installer, cinch to install the distribution. Sidekick is text based so you don’t have to worry about video cards and monitors (makes things simpler – remember this is a server oriented distribution). This can greatly speed the configuration.

The packages that Caos NSA has by default are very useful for servers but also have a cluster flavor. For example, it has (as of the time of this writing), OFED 1.2 (OFED = Open Fabrics Enterprise Distribution) which gives Infiniband support common in HPC), Open MPI 1.2.5, Slurm 1.2.19, gcc 4.1.2, and environment modules, Cports (Caos source based/ports packaging system integrated with environment modules and commercial compiler profile support, and others.

Caos NSA has other features that many other distributions don’t have. One important feature is that is uses a very recent kernel – While some people don’t like to work with recent kernels because they want many more people to test it before they use it, the recent kernels have features or capabilities for recent chipsets and hardware as well as new features in the kernel itself. While I understand the desire for more “tested” kernels, many times these kernels cannot use recent hardware because it’s not supported. In addition, the kernel can be an exciting place with lots of new developments that can help HPC performance. So, I think having a recent kernel is important.

One feature that Caos NSA has that most Redhat based distributions don’t have, is support for XFS. XFS is important for clusters because it is arguable one of the fastest file systems, particularly for large files. This is particularly important for codes that require lots of local file system performance.

Caos NSA also configures the Ethernet interfaces for IPv6, not just IPv4. While I’m not using IPv6 at the moment, I think a distribution that can configure IPv6 at this point is far ahead of others.

A fourth feature that I think is potentially important is a firewall. While many distributions have firewall capability, I’ve found installing them to be less than easy. But in Caos NSA setting up a firewall for any and all network interfaces is very easy. You tell it which Ethernet interfaces you want to have a firewall and it automatically configures the firewall and then you can open certain ports (as desired). But, Caos doesn’t make any guarantees on the firewall for obvious legal reasons.

Finally, Caos NSA isn’t just a Linux distribution but is also tightly integrated with Perceus – a fantastic Cluster Management System (CMS).


In last month’s column I talked about building and installing Perceus with Fedora Core 8. In that column I discussed how to build it from scratch, how to configure it, and how to get a small 2 node cluster working. In the case of Caos NSA, Perceus is already tightly integrated.

As part of the installation, Caos NSA asks if you want to install Perceus. If you want this (and who doesn’t) it will install it and configure it for you after you answer a couple of questions. It will ask you what network interface you want to use for the cluster (this should be a private network). Then it will ask you what range of IP addresses you want to use and a couple of other simple questions, and bingo, Perceus is configured and ready to go.

At this point, you can follow the same basic steps as in the previous column. You should download a VNFS capsule, import it into Perceus, boot some nodes, and point the imported capsules at these new nodes.

Now that we have the basics of what Perceus is doing, let’s actually install Caos NSA and Perceus on a small two node cluster to walk through the process.

Caos NSA Installation and Configuration

I’m working with a small two node cluster to demonstrate Caos NSA and Perceus. The two nodes are basic AMD Opteron nodes. The master node has a single IDE drive in it (80 GB) and three NICs: (1) a fast Ethernet NIC to the outside world (IP:, (2) an Intel GigE NIC (IP, and (3) a DLINK fast Ethernet NIC for management and storage (IP I downloaded the Caos NSA image to a DVD and booted from that DVD.

To start the installation I typed,

install ipaddr= netmask= gw= \
  ifdev=eth0 fs=xfs ns=xx.xxx.xxx.xx hostname=home64 \
  layout=bigsrv disk=hda

and Caos NSA takes off. The variable ipaddr is the IP address of the master node, gw is the gateway (in this case my home router), ifdev is the ethernet device it should use for the installation, fs is the file system I want to use (thank God Caos NSA has xfs), ns is the nameserver which I’ve left blank to protect the innocent, layout is the particular file system layout I wanted (other options are “manual” “basic” and a few others).

The installation then starts installing Caos NSA that installs the kernel ( – thankfully a newer kernel), configured grub, and a few other items, and then reboots. After the reboot sidekick starts up to help you finish the installation (remember than sidekick is also an admin tool that you can use after the installation). The first screen that pops up is something of a registration request that is really optional.

The next few steps deal with some details about the installation. Here is the list of steps I took up to the point of configuring Perceus.

Caos NSA Installation steps:

  1. Registration
  2. Root password
  3. Time zone configuration
  4. NIC configuration (eth0, eth1, and eth2 in my case)
  5. Firewall configuration (including opening ports)
  6. System Profile (what basic configuration do you want)
    • Desktop
    • Cluster

The last step is to actually install the packages for the requested configuration. In this case I chose to install Caos NSA as both a desktop and a cluster (the desktop configuration installs XFCE and a basic VESA driver for those of use who like to use a web browser on the master node such as me).

The next step in the installation asks if you want to configure Perceus. After answering yes, the next steps are fairly simple:

Comments on "Caos NSA and Perceus: All-in-one Cluster Software Stack"

I do not even know the way I finished up right here, but I believed this submit used to be good. I don’t understand who you might be but definitely you’re going to a well-known blogger in the event you are not already. Cheers!

Just want to say your article is as astounding. The clarity in your post is simply cool and i could assume you’re an expert on this subject. Well with your permission allow me to grab your feed to keep up to date with forthcoming post. Thanks a million and please carry on the gratifying work.

HI Buddy’s It is a Good post but I was wondering if you could write a litte more on this subject? I’d be very thankful if you could elaborate a little bit further. Appreciate it!loveshhuda full movie download

I was very impressed by this post, this site has always been pleasant news Thank you very much for such an interesting post, and I meet them more often then I visited this site.section 80g

One of our visitors lately proposed the following website.

Below you?ll uncover the link to some web sites that we consider you need to visit.

Excellent post however I was wanting to know if you could write a litte more on this subject? I’d be very grateful if you could elaborate a little bit further. Thank you!

I love what you guys are usually up too. Such clever work and coverage! Keep up the superb works guys I’ve added you guys to our blogroll.

Watch Pacquiao vs Bradley Live Stream . Pacquiao vs Bradley 3 Live Streaming on April 9 at MGM Grand Arena in Las Vegas Nevada, USA

Very few web sites that happen to be comprehensive beneath, from our point of view are undoubtedly effectively really worth checking out.

Fantastic article.Thanks Again. Cool.

Great post! We will be linking to this great content on our website. Keep up the great writing.

Designers of the times such as Dior and Givenchy went away from the traditional towards new beginnings introducing shapes that emphasised the silhouette of a woman’s shape, rather than boning clothing in the uncomfortable manner of the 40s, realising that women wanted glamour though also wanted functional clothing that was more comfortable and easier to wear. Stop fretting about what this younger guy sees in you, and what others might think and enjoy the attention. They idolize a woman who is not threatened when he does not phone the following day, a lady who maintains a social life and isn’t totally absorbed in him, as well as a lady that has her own interests.

Here is a great Weblog You may Obtain Interesting that we encourage you to visit.

Very informative article post.Thanks Again.

Just beneath, are many entirely not related websites to ours, even so, they are surely really worth going over.

Right now it appears like WordPress is the best blogging platform out there right now. (from what I’ve read) Is that what you’re using on your blog?

Great blog you’ve got here.. It’s hard to find high-quality writing like yours these days. I truly appreciate individuals like you! Take care!!

This is a topic that’s close to my heart… Thank you! Where are your contact details though?

Its like you read my mind! You appear to know a lot about this, like you wrote the book in it or something. I think that you can do with some pics to drive the message home a bit, but instead of that, this is magnificent blog. A great read. I will definitely be back.

There will be a special screening of the award winning entries at the Charles Wood Theaterin Glens Falls, NY, on January 22, 2011. The patterns look like something that was designed via a graphing calculator or a Game Boy. ‘ Resolution will be the amount of pixels on a display screen, and together with invigorate rate they choose picture quality.

It is in reality a great and useful piece of info. I’m glad that you simply shared this useful info with us. Please keep us up to date like this. Thanks for sharing.

Greate post. Keep posting such kind of info on your site. Im really impressed by your site.
Hello there, You’ve done an incredible job. I will certainly digg it and in my opinion suggest to my friends. I am confident they’ll be benefited from this website.

Hi there, after reading this remarkable post i am also glad to share my know-how here with colleagues.

Tremendous issues here. I’m very happy to peer your article. Thank you a lot and I am looking forward to contact you. Will you kindly drop me a e-mail?

Very few websites that come about to be in depth beneath, from our point of view are undoubtedly well worth checking out.

Although web-sites we backlink to beneath are considerably not connected to ours, we really feel they are truly worth a go by way of, so possess a look.

Great blog here! Also your site loads up fast! What web host are you using? Can I get your affiliate link to your host? I wish my website loaded up as fast as yours lol

That could be the finish of this post. Here you will find some web pages that we think you will enjoy, just click the hyperlinks.

Thanks for finally talking about >Caos NSA and Perceus: All-in-one Cluster Software Stack <Loved it!

vente cialis cialis pas cher
cialis cialis
cialis pas ?her acheter cialis
cialis sin receta comprar cialis
cialis generico cialis generico

I’d like to find out more? I’d love to find out some additional information.

I would like to thank you for the efforts you have made in writing this article. I am hoping the same best work from you in the future as well.??? ??????

I would like to thank you for the efforts you have made in writing this article. I am hoping the same best work from you in the future as well.??? ??????

We prefer to honor many other net sites on the web, even though they aren?t linked to us, by linking to them. Below are some webpages really worth checking out.

This is a topic that is close to my heart… Cheers! Where are your contact details though?

That will be the finish of this post. Right here you?ll discover some internet sites that we think you?ll enjoy, just click the hyperlinks.

Please take a look at the web sites we comply with, which includes this 1, because it represents our picks through the web.

Hey! I know this is somewhat off topic but I was wondering which blog platform are you using for this website? I’m getting sick and tired of WordPress because I’ve had issues with hackers and I’m looking at options for another platform. I would be awesome if you could point me in the direction of a good platform.

Canelo vs Khan Canelo vs Khan live stream on HBO Pay Per View. Collect Canelo vs Amir Khan Boxing Now and canelo vs amir khan live stream watch Canelo vs Amir Khan Live Stream Fight live on May 7 in Las Vegas Nevada, T-Mobile Arena.

Watch Canelo vs Khan The Biggest Fight in This Year 2016 & Canelo vs Amir khan Are You Want To Canelo vs Amir Khan Live Stream Stay With With Us And Save This Event live on May 7,2016.

If you are going for finest contents like me, only pay a visit this web page daily as it provides feature contents, thanks

Wonderful story, reckoned we could combine a number of unrelated data, nonetheless definitely really worth taking a look, whoa did 1 study about Mid East has got a lot more problerms too.

I am sure this paragraph has touched all the internet people, its really really good post on building up new blog.

I would like to thank you for the efforts you have made in writing this article. I am hoping the same best work from you in the future as well. http://www.hcgshotsdiscount.com/ hcg

I just want to say I’m beginner to blogs and honestly liked your website. Almost certainly I’m planning to bookmark your blog . You absolutely come with beneficial writings. Thanks for sharing your website.

I’m curious to find out what blog platform you happen to be working with? I’m experiencing some minor security issues with my latest website and I’d like to find something more secure. Do you have any recommendations?

excellent issues altogether, you just gained a logo new reader. What would you recommend about your submit that you simply made a few days in the past? Any sure?

It’s remarkable to visit this website and reading the views of all mates about this post, while I am also keen of getting experience.

Leave a Reply