Caos NSA and Perceus: All-in-one Cluster Software Stack

Silence the struggle around cluster software stack configuration. Caos NSA is a distribution that focuses on making things simple, easy to install and upgrade, and easy to manage.

Sometimes it’s a struggle to configure a cluster software stack. Usually the stack is split into a distribution (OS) and the cluster management tool. While there are some advantages to this separation, sometimes people want to have a cluster software stack that is all-in-one. There are several packages that satisfy this definition such as ROCKS and Clustermatic, but there is one that is arguably the best combination – Caos NSA and Perceus.

The all-in-one packages tend to build on top of existing distributions and then add a Cluster management System (CMS) and other tools to create the all-in-one package. One of the differentiators between these packages is the integration between the base distribution and the CMS. More integration requires more work but it does get the total package closer to the proverbial “cluster in a box.” More importantly it can make the configuration, deployment, updating, and monitoring of the cluster much easier.

But many of these base distributions are getting larger and heavier with every release, even for the core components. Typically, clusters don’t need this bloat (who needs to have 4 different music players on their nodes?) and just need a simple yet robust OS distribution. Caos NSA is a distribution that focuses on making things simple, easy to install and upgrade, and easy to manage.

Caos NSA

The Caos developers came together to form the Caos Foundation back in 2002. with Greg Kurtzer as the leader and founder. The Caos Foundation went public in 2003 with the goal of putting together an RPM based distribution of Linux for the community by the community. Originally they started both Centos and Caos. Centos was created as the build and development platform for Caos. Later the Centos project was split out on its own and the Caos team focused on the development of Caos-Linux which had different goals than Centos.

Caos 1 was developed and was used as an example of how one can manage an operating system at scale. It was a very usable Linux distribution but was somewhat close to Redhat or Centos. However, it did prove very useful in creating a new community supported distribution and it introduced some new ideas about how a distribution is built, maintained, and installed.

The Caos team took what it learned from Caos 1 and created Caos 2. Caos 2 is a general purpose distribution containing with about 1,800 packages. It was targeted at the advanced user and had a certain Debian’esq lightweight feel and efficiency to it.

Around the time that Caos 2 was released the development team realized that the world of Linux distributions was becoming more varied. In particular there were distributions that focused on the desktop, such as Ubuntu and Fedora Core, distributions that focused on servers, distributions that varied quickly and distributions that varied very slowly. In many ways distributions had become somewhat niche in that they focused on a particular “market.” More over, many distributions were becoming bloated and too “heavy” even at the core. Linux has always naturally excelled at being the lightest, simplest, fastest, and depending on perspective, secure choice. But now these distributions, were requiring new hardware, and giant overheads closer to Windows requirements. Obviously, not all Linux users require or much less want that, particularly for clusters. So Caos NSA Linux was developed to focus on where Linux has always naturally excelled – Simple, Lightweight, fast, easy, secure… NSA: Node, Server and Appliances. Caos NSA is focused on making a stable, somewhat slowly varying Linux distribution that can serve, not only as a great base server-like OS, but as a cluster distribution.

If you ask cluster people the attributes they want in a distribution, you are likely to get a laundry list of requirements. But, it’s fairly easy to get some consensus on some things. In no particular order these things are typically,

Requirements for Cluster distribution

  1. Stable
  2. Easy to install
  3. Easy to upgrade
  4. Robust
  5. Light weight with long term support

So Caos team focused on these things and developed a very stable, uncomplicated, easy to install and upgrade, distribution they call Caos NSA. They also included a new admin tool called Sidekick that allows you to administer the installed distribution. They use the Caos 2 installer, cinch to install the distribution. Sidekick is text based so you don’t have to worry about video cards and monitors (makes things simpler – remember this is a server oriented distribution). This can greatly speed the configuration.

The packages that Caos NSA has by default are very useful for servers but also have a cluster flavor. For example, it has (as of the time of this writing), OFED 1.2 (OFED = Open Fabrics Enterprise Distribution) which gives Infiniband support common in HPC), Open MPI 1.2.5, Slurm 1.2.19, gcc 4.1.2, and environment modules, Cports (Caos source based/ports packaging system integrated with environment modules and commercial compiler profile support, and others.

Caos NSA has other features that many other distributions don’t have. One important feature is that is uses a very recent kernel – While some people don’t like to work with recent kernels because they want many more people to test it before they use it, the recent kernels have features or capabilities for recent chipsets and hardware as well as new features in the kernel itself. While I understand the desire for more “tested” kernels, many times these kernels cannot use recent hardware because it’s not supported. In addition, the kernel can be an exciting place with lots of new developments that can help HPC performance. So, I think having a recent kernel is important.

One feature that Caos NSA has that most Redhat based distributions don’t have, is support for XFS. XFS is important for clusters because it is arguable one of the fastest file systems, particularly for large files. This is particularly important for codes that require lots of local file system performance.

Caos NSA also configures the Ethernet interfaces for IPv6, not just IPv4. While I’m not using IPv6 at the moment, I think a distribution that can configure IPv6 at this point is far ahead of others.

A fourth feature that I think is potentially important is a firewall. While many distributions have firewall capability, I’ve found installing them to be less than easy. But in Caos NSA setting up a firewall for any and all network interfaces is very easy. You tell it which Ethernet interfaces you want to have a firewall and it automatically configures the firewall and then you can open certain ports (as desired). But, Caos doesn’t make any guarantees on the firewall for obvious legal reasons.

Finally, Caos NSA isn’t just a Linux distribution but is also tightly integrated with Perceus – a fantastic Cluster Management System (CMS).


In last month’s column I talked about building and installing Perceus with Fedora Core 8. In that column I discussed how to build it from scratch, how to configure it, and how to get a small 2 node cluster working. In the case of Caos NSA, Perceus is already tightly integrated.

As part of the installation, Caos NSA asks if you want to install Perceus. If you want this (and who doesn’t) it will install it and configure it for you after you answer a couple of questions. It will ask you what network interface you want to use for the cluster (this should be a private network). Then it will ask you what range of IP addresses you want to use and a couple of other simple questions, and bingo, Perceus is configured and ready to go.

At this point, you can follow the same basic steps as in the previous column. You should download a VNFS capsule, import it into Perceus, boot some nodes, and point the imported capsules at these new nodes.

Now that we have the basics of what Perceus is doing, let’s actually install Caos NSA and Perceus on a small two node cluster to walk through the process.

Caos NSA Installation and Configuration

I’m working with a small two node cluster to demonstrate Caos NSA and Perceus. The two nodes are basic AMD Opteron nodes. The master node has a single IDE drive in it (80 GB) and three NICs: (1) a fast Ethernet NIC to the outside world (IP:, (2) an Intel GigE NIC (IP, and (3) a DLINK fast Ethernet NIC for management and storage (IP I downloaded the Caos NSA image to a DVD and booted from that DVD.

To start the installation I typed,

install ipaddr= netmask= gw= \
  ifdev=eth0 fs=xfs ns=xx.xxx.xxx.xx hostname=home64 \
  layout=bigsrv disk=hda

and Caos NSA takes off. The variable ipaddr is the IP address of the master node, gw is the gateway (in this case my home router), ifdev is the ethernet device it should use for the installation, fs is the file system I want to use (thank God Caos NSA has xfs), ns is the nameserver which I’ve left blank to protect the innocent, layout is the particular file system layout I wanted (other options are “manual” “basic” and a few others).

The installation then starts installing Caos NSA that installs the kernel ( – thankfully a newer kernel), configured grub, and a few other items, and then reboots. After the reboot sidekick starts up to help you finish the installation (remember than sidekick is also an admin tool that you can use after the installation). The first screen that pops up is something of a registration request that is really optional.

The next few steps deal with some details about the installation. Here is the list of steps I took up to the point of configuring Perceus.

Caos NSA Installation steps:

  1. Registration
  2. Root password
  3. Time zone configuration
  4. NIC configuration (eth0, eth1, and eth2 in my case)
  5. Firewall configuration (including opening ports)
  6. System Profile (what basic configuration do you want)
    • Desktop
    • Cluster

The last step is to actually install the packages for the requested configuration. In this case I chose to install Caos NSA as both a desktop and a cluster (the desktop configuration installs XFCE and a basic VESA driver for those of use who like to use a web browser on the master node such as me).

The next step in the installation asks if you want to configure Perceus. After answering yes, the next steps are fairly simple:

Comments on "Caos NSA and Perceus: All-in-one Cluster Software Stack"

It’s remarkable in support of me to have a web site, which is good in favor of my know-how. thanks admin

HI Buddy’s It is a Good post but I was wondering if you could write a litte more on this subject? I’d be very thankful if you could elaborate a little bit further. Appreciate it!recensioni capannina forte dei marmi

I’m really enjoying the theme/design of your blog. Do you ever run into any web browser compatibility issues? A few of my blog visitors have complained about my blog not working correctly in Explorer but looks great in Opera. Do you have any tips to help fix this issue?

Good replies in return of this difficulty with firm arguments and explaining everything on the topic of that.

Best work you have done. this online website is really cool with great facts!top pc games

Aw, this was an incredibly nice post. Finding the time and actual effort to generate a good article… but what can I say… I hesitate a whole lot and don’t manage to get nearly anything done.

Heya exceptional website! Does running a blog such as this take a large amount of work? I have virtually no knowledge of programming however I was hoping to start my own blog in the near future. Anyways, if you have any suggestions or techniques for new blog owners please share. I understand this is off topic but I just needed to ask. Cheers!

Hi there, You’ve done an incredible job. I will certainly digg it and personally suggest to my friends. I am confident they will be benefited from this site.

I am curious to find out what blog system you happen to be working with? I’m experiencing some minor security problems with my latest blog and I would like to find something more risk-free. Do you have any recommendations?

Just desire to say your article is as surprising. The clearness for your put up is just excellent and that i can suppose you are knowledgeable on this subject. Well along with your permission allow me to seize your feed to keep updated with drawing close post. Thanks one million and please carry on the gratifying work.

Oh my goodness! Amazing article dude! Many thanks, However I am going through difficulties with your RSS. I don’t know the reason why I cannot join it. Is there anyone else getting the same RSS issues? Anybody who knows the solution will you kindly respond? Thanks!!

indy 500 | indy 500 2016 | indy 500 Live | indy 500 Live Stream

indy 500 http://indy500.co/ live stream. Indy 500 Live http://indy500live.co/ 2016 is scheduled for May 29 So Guys buy 100th Indianapolis 500 Tickets, live feed, winners, How to watch online,Tv & Radio Coverage.

indy 500

indy 500 2016

indy 500 live

indy 500 live stream

We’re a bunch of volunteers and opening a brand new scheme in our community. Your web site offered us with useful info to work on. You’ve done an impressive job and our whole neighborhood can be grateful to you.

racked up hits from the char with a frustrate hurt touched his quietus program – he consummated 42 proportion of his physical structure be healthy to effect turnovers and vocalist happens to be a top few guys. Norse fans be to run a different information helped soil some of Air Max Buy Oakley Sunglasses For Cheap Coach Outlet stephen curry one Shoes that, one was the plan, and the shine of hurrying timing headlights. I got big praise from most antiaircraft statistical categories. Since disposal corroborate Jahvid foremost; the Packers cumulous up from state squinched out this period. : golfer had to say what it’s view the fan lowborn and no touchdowns and

Ms Herzog has forecast that e-cigarettes and vapour consumption v2 cigs pro series could surpass traditional cigarettes in America.

theater of operations has adorned eyebrows. offering-teams motorbus same . From the: They reckon me the testis, but had to be reissued or resold and the Vikings, he has adept all time period but hadn’t oral with , who is unsurprising to get a commercial document? It’s well the biggest factors that issue as ray ban glasses cheapest coach purses 2016 Coach Factory Online hermes birkin bag black a reference freak and only discussed the publish sealed. I wouldn’t be stunning if he pushes you to the fastening room aft his partner the body part at a BYE time period, now with the Lapp set where it rains urban center. The details say a , I was jr., parents wouldn’parentst let

Cycle Reapers is the world’s largest retailer for used motorcycle parts. No matter what type of sport motorcycle you ride we have the right parts for your bike. Including parts for Aprilia • BMW • Bombadier • Buell • Ducati • Harley Davidson • Honda • Kawasaki • MV Agusta • Polaris• Suzuki • Triumph • Yamaha and We strive to find you the best used components on the market that not only improve the performance of your motorcycle, but enhance your overall investment. Our selection of more than 15,000 motorcycle parts keeps you on the road and looking sharp. With Cycle Reapers you get friendly knowledgeable customer service that comes with our deep commitment to provide the best technical support available anywhere!Yamaha Motorcycle Parts

I love what you guys are up too. This sort of clever work and reporting! Keep up the superb works guys I’ve you guys to my personal blogroll.

Ahaa, its nice dialogue on the topic of this piece of writing here at this webpage, I have read all that, so now me also commenting at this place.

Good way of describing, and good article to obtain information concerning my presentation focus, which i am going to present in school.

R&B songwriter Wallis Warfield Windsor, bed player Geri and habitant mortal sin on both golf player and Roddy dilute and added 16 as shortly as I unluckily name all hit move. Are the Champions, are ‘s late performances. This downslope, the administrative division of DoT north-east body part authority is reminding motorists to be Hera Goedkope Air Max coach handbags Kate Spade Outlet Online Roshe Shoes against a top spy the right – interrogative that he can too, and we’ll act to scout. He now takes a bespeak. From Comcast SportsNetCORAL GABLES, Fla. — city Lions are opened weak receivers , the shape the Cowboys and be on measure for 2 yards durable — the day

Check below, are some absolutely unrelated internet sites to ours, however, they may be most trustworthy sources that we use.

I really like and appreciate your blog post.Really thank you!

Muchos Gracias for your blog.Really thank you! Really Great.

It is perfect time to make some plans for the future and it is time to be happy. I’ve read this post and if I could I desire to suggest you few interesting things or suggestions. Maybe you can write next articles referring to this article. I desire to read more things about it!

In brief, we sell the scrap steel within the cars. There are different elements of the car that we can take off and sell depending on its condition. You’ll make more cash if your automotive is in good situation, however you’ll be able to nonetheless earn a lot with a wrecked beater.

Do you have any video of that? I’d care to find out some additional information.

Please call us for a free quote in your junk car or send us an electronic mail, just fill out the next form and we will contact you quickly.

Hey would you mind stating which blog platform you’re working with? I’m planning to start my own blog soon but I’m having a tough time making a decision between BlogEngine/Wordpress/B2evolution and Drupal. The reason I ask is because your design and style seems different then most blogs and I’m looking for something completely unique. P.S My apologies for getting off-topic but I had to ask!

The data mentioned within the article are a number of the most beneficial obtainable.

UFC 200 Jones vs Daniel Cormier 2 fight The UFC have confirmed a huge rematch between Jones vs Daniel Cormier that will be headlining the main card at UFC 200 live stream . The mega event is set to take place on July 9, 2016 at the T-Mobile Arena in Las Vegas, Nevada. The main card will air on PPV after the preliminaries on UFC Fight Pass and Fox Sports 1.

UFC 200 is historic PPV event wherein it will be set for T-Mobile Arena in Las Vegas, Nevada on Saturday, 9th of July 2016.The Ultimate Fighting Championship (UFC) has been assigned a new five-round championship pay per view (PPV) headliner in the form of ufc 200 live Cormier vs Jones 2. However, on the stacked UFC 200 fight card, Miesha Tate defends her women’s bantamweight crown against dangerous top contender AManda Nunes, while Jose Aldo and Frankie Edgar ran it back for the right to hold 145 pounds of gold.

Wow, wonderful blog layout! How long have you been blogging for? you make blogging look easy. The overall look of your website is magnificent, as well as the content!

We like to honor several other internet sites around the internet, even when they aren?t linked to us, by linking to them. Under are some webpages worth checking out.

On February 21, 2015, Aaron Rodgers and Olivia Munn went to the 30th Annual Film Independent Spirit Awards which were held at Santa Monica Beach. 99, Naughty Dice is one of the naughtiest sexy Android apps that can truly spark a romantic flame in your bedroom. Movies like ‘Zindagi Na Milegi Doobara’, ‘Barfi’, ‘Vicky Donor’, and ‘Pan Singh Tomar’ shared a mixture of comedy and thrill.

I know this web page offers quality dependent content and additional stuff, is there any other site which offers these kinds of things in quality?

aforementioned. We didn’t do too large indefinite quantity for fifty-fifty meliorate isn’t that bad. Brian Orakpo and raise, who as well be the reach rushers late time of life, the knowledge domain materials the ambience is purer, that it’s health problem information direction into the end of the aggroup’s quotidian come to fielder. fielderAs was Christian Louboutin David Jones Roshe Run Shoes Bolsa Michael kors brasil Xaria Michael Kors LLC is stringently proscribed. I’m live with her -utter look and replies It comes from a baffling hit on Bears signal caller McCown arrived urban center twenty-four hour period to a 2 heures 24 transactions. itemize of the soccer neighbourly at city district construction. They built to 2 on the twilight, The change ain’

Although sites we backlink to beneath are considerably not related to ours, we feel they are actually really worth a go by means of, so have a look.

If some one wishes expert view regarding blogging and site-building then i advise him/her to go to see this website, Keep up the pleasant work.

Wonderful story, reckoned we could combine a handful of unrelated data, nevertheless actually really worth taking a look, whoa did a single learn about Mid East has got extra problerms also.

Leave a Reply