Cluster Tools for the Enterprise

Enterprise-level clusters? It's time we start thinking about them.

The Cry from the Wilderness

For whatever reason lately I’ve become sensitive to getting clusters to be considered “Enterprise-ready.” I’ve been talking to various people who either have clusters that are managed outside the mainstream Enterprise or who are new to clusters and want to find a way to get the clusters into the IT shop (so they don’t have to manage them). But a common theme from both sides is that Cluster Tools are missing what be termed “Enterprise Class” tools.

So how do we make existing Cluster tools into Enterprise-Class Cluster Tools? One approach is to add the pieces that are missing. I think there are some pieces that can easily be added to existing tools or just better installation and management tools to make them more Enterprise ready. However, I think there are some tools that need to be developed. Regardless let’s look at some things that are needed to move Cluster Tools into the Enterprise realm.

Bare Metal Backup

One of the key components that the Enterprise IT groups require in servers is a bare metal backup. This can be thought of a Disaster Recovery (DR) requirement. The idea is that if the server goes down and either a new OS drive is required or a complete new server is required, you can quickly restore it to the originally configured state without having to work through the installation, configuration, etc. that you originally had to do. It’s fairly easy to see the usefulness of a bare metal backup.

A bare metal backup is a bit different than an image. With an image you will need the same drive(s) as you had before. With a backup you don’t necessarily have to have the same drives as before. They have to be at least the same size, but not necessarily the same type or the same capacity. However, to make life easier it’s probably a good idea to have the drive(s) be the same type as before.

A bare metal backup for a cluster only applies to the critical components of a cluster. This includes the head node, any login nodes, IO nodes, and possibly storage nodes (for example, IO nodes for a parallel file system such as Lustre, GPFS, or GFS). Basically you create bare metal backups for anything but the compute nodes. Typically people think of the compute nodes as disposable. That is, if a compute node fails you just replace it and restart the job that was using the node. This also means that you don’t place anything critical on the node. For example, if you put the OS or a subset of the OS in a ramdisk in the compute node then if there is a hard drive in the node, it can be used for local storage or for swap space.

Using bare metal backups in practice is fairly easy, but as will all backups, requires some discipline. When a cluster is first configured and tested, but perhaps before it goes into full production, the first bare metal backup should be made of the head node, the login nodes, and possibly storage nodes (not the storage itself but the OS on the storage nodes). Then if any configuration changes are made to any of these nodes, a new bare metal backup is made of the entire set of nodes. These backups should be get a safe place and even better, extra copies are made and kept off-site (a simple safety deposit box can be used for this).

To make the process of creating bare metal backups easier it should be integrated with the cluster management tool. Then when a configuration change is made a bare metal backup can be created. This is also true for any firmware or BIOS upgrades. However, you will need to keep track of the current firmware versions so any replacement hardware is configured with identical firmware versions. In addition, if the image for the nodes is changed, the cluster management tool can prompt the administrator to make the backup.

One of my favorite tools for creating a bare metal backup tool called Mondorescue (www.mondorescue.org). It’s a GPL licensed bare metal backup tool that is also scriptable. It has a variety of ways to store the backups including NFS and is very easy to use.

Reporting Tool

Another enterprise class tool that is absent from virtually every cluster that I know of is a reporting tool. A reporting tool takes information about the system and creates a summary report. This report is very useful to get an idea at a reasonable high level but with some detail, as to how the system was used. The report can be a file or a delivered via the web or embedded in applications. it can include tables (including a spreadsheet), or graphs.

These reports can be used for visualization and reporting on the cluster. For example, it can contain information such as the number of jobs run in a certain time period, total uptime of the nodes, number of nodes down, the total down time for the cluster (sum of the down time of the nodes that were down), file space usage, log in times, administration time, any security information, such as failed login attempts, on the head node, any network traffic information you have gathered, and just about anything else you can measure or are interested in seeing a report.

I’m sure there are some people who are reading this column and asking if I’m on crack. Well the obvious answer is no and it’s not the coffee either. Let me explain why I think reports are a very useful and really a required feature for enterprise class clusters.

I think clusters are cool in and off themselves, but they are very useful for solving large scale problems. Many times, people get a cluster up and running and declare victory and start using it. However, as they will find out, that getting it up and running is only half the problem. The other half is keeping the system running. Simple day to day activities on the cluster such as watching the queues, killing hung jobs, starting jobs that need to be started, juggling project priorities, adding users, removing users, recovering or restoring data, debugging nodes that are down, bringing up repaired nodes, restarting hung nodes, and fixing user problems (“why won’t my script work?”) can take a great deal of time, particularly when you have a number of users with varied experience running on HPC class systems. Good administrators will fix these problems but the better administrators will try to determine why they happened in the first place and how to make sure they don’t happen again. The best administrators won’t necessarily through up barriers to prevent users from doing things but will give users better tools and better education to help them solve their problems.

If an administrator had a way to summarize activity on the cluster and correlate that with problems, then it’s much easier to see cause and effect and take corrective action. I know this sounds awfully “enterprise” like and some people are cringing and wanting to strangle me, but the fact of the matter is that having this information can make it easier to understand what is happening in the cluster and fix the problem. The enterprise world uses the phrase “business intelligence.” While I’m not hot on that phrase (it does have an oxymoron ring to it), it does hint at what you’re trying to do — more efficient use of the resources and making your cluster run more smoothly.

Comments on "Cluster Tools for the Enterprise"

Great article, I would love to read more of your things. Maybe it is possible to read mine https://www.facebook.com/posicionamiento.seo.chile

Terrific post!!! This webpage simply keeps me coming back. I enjoyed the energy which you put into it as well as fancy browsing all of the posts. Hi and thanks a bunch and also keep the brilliant work up!!!

Awesome blog! Do you have any suggestions for aspiring writers? I’m planning to start my own site soon but I’m a little lost on everything. Would you suggest starting with a free platform like WordPress or go for a paid option? There are so many choices out there that I’m completely overwhelmed .. Any recommendations? Many thanks!

You will find thousands of sites that requires comments on them. What is the purpose of blog page comments?.

Basica?l?, you can only do 5 invoices, 5 pr?posals, and 5 expenses in the free version. If
yo? ha?e r?ceived IRS tax notice, yo? need
not to be panic. For cond?cting the additional analysis on the specific CPA network, which will assist you to earn fund?, all you want to
do is you will need to ?lace in the identify of that network along ?ith some other appli?a?le
keyword phrases that cou?d possibly assist yo? in discov?ring ?ut ?f
there ?s any precis? chal?enge with t?at individu?l or?anization.

Here is my web bl?g … John Costello John Miller Boca Raton CPA

Good info. Lucky me I discovered your website by accident
(stumbleupon). I’ve saved it for later!

Yourself contain acquired some very seriously valuable information written below. Great undertaking and hold short article superior stuff.

Thank you for another informative site. Where else could I get that type of info written in such an ideal way? I’ve a project that I am just now working on, and I’ve been on the look out for such information.

Motivation is the key to positivity in our lives. If we were not motivated enough, we would just not have the will in us to go about our daily lives. Its motivation that helps us get through the most mundane things– motivation for working harder, motivation to have a healthy relationship, motivation to earn more, motivation to have a happy family. And yet sometimes we find ourselves lacking in motivation; I think we have all had days when getting out of bed to get ready to go to work seemed like a Herculean task. Sometimes lack of motivation can really bring people down, and hinder their efficiency and ability. This may lead to frustrations and further breakdown of communication between people. If you intercept this lack of motivation timely, you have help at hand. Go for hypnotherapy for motivation to get the zest in life back. Hypnotherapy is intrinsically related to the concept of motivation, and can help you in every walk of life. From motivation in sports, to business; from quitting smoking to losing weight, hypnotherapy for motivation could help you out. Hypnotherapy for motivation works towards removing those mental blockages by connecting with your subconscious.Hypnotherapy is based on the formula of auto suggestion, and it works on deeper levels than one can imagine. Hypnotherapy for motivation converses with your subconscious mind, and urges it to alter self-perception and self– image.In my practice at Oxfordshire, I have helped clients deal with a variety of issues through hypnotherapy for motivation. The underlying theory of hypnotherapy is based on motivation, and we need it in every walk of life. Hypnotherapy for motivation lets you focus on the particular task at hand, however if the task is undefined, hypnotherapy will be rendered ineffective.Hypnotherapy for motivation can seem to be too simple a solution for these problems; however you will be surprised at their effectiveness. Through altering self-perception and self-image, hypnotherapy gives you a whole new view of the world. Define your goals now, and dive into the vast sea of your subconscious mind through hypnotherapy.

Most of whatever you point out is astonishingly accurate and that makes me wonder why I had not looked at this with this light previously. This piece really did turn the light on for me personally as far as this specific topic goes. Nevertheless at this time there is actually one particular factor I am not necessarily too cozy with and while I attempt to reconcile that with the central idea of your position, permit me see just what all the rest of your visitors have to say.Very well done.

Youve obtained a superior offer of wording for only taking a single or two visuals. Probably on your own could area it out better?

This is always a outstanding world wide web, could on your own be intrigued within just doing work on an job interview pertaining to particularly how on your own produced it? If so email messages me!

Due for sharing.. Just stumbled on your world wide web When executing a Google glance. Suitable information and facts BTW

I have been browsing online more than 3 hours today, yet I never found any interesting article like yours. It is pretty worth enough for me. In my view, if all webmasters and bloggers made good content as you did, the web will be a lot more useful than ever before.

I believe you have mentioned some very interesting points, thankyou for the post.

Cuando equiparas coches de alquiler en Aeropuerto de Barna, lo que hacemos es mostrarte las mejores ofertas en una amplia gama de automoviles a fin de que puedas escoger el mejor vehiculo a un coste que se adapte a tu presupuesto. Las Tarifas son bastante accesibles, un recorrido dentro de la zona de Playa del Ingles/Maspalomas por poner un ejemplo, por norma general no llega a cinco â?¬. Cuando comparas vehiculos de alquiler en Aeropuerto de Gran Canaria Las Palmas, lo que hacemos es mostrarte las mejores ofertas en una extensa gama de vehiculos para que puedas elegir el mejor vehiculo a un precio que se adapte a tu presupuesto.

It’s actually a nice and useful piece of information. I’m glad that you shared this useful information with us. Please keep us informed like this. Thanks for sharing.

Wonderful post! This site just keeps me coming back. I admire the work which you put into it as well as appreciate reading through all the articles. Hi and thanks a whole lot as well as keep the fantastic work up!!

I do love the way you have framed this matter plus it does supply me some fodder for thought. On the other hand, from everything that I have personally seen, I simply hope when other responses pack on that men and women keep on issue and in no way start on a soap box involving the news du jour. Anyway, thank you for this fantastic point and even though I do not necessarily concur with this in totality, I respect the viewpoint.

Find the best Dog Daycare in McLean Virginia at the Capitol Canine Club. Specializing in Doggie Day Care and Dog Boarding in McLean Virginia

You c?n r?all? help most of your clients by helping them use financial statements as
a ba?is for making better business decisions. They will
do al? the talking for yo? w?t? t?e agent.
Av?id hype or gimmicks and let peo?le know exactly what t? expect and
what they are requi?ed to do in order to receive what they are being offered.

Lo?k into my web site; Boca Raton CPA supplies canada

This kind of lovely blog you’ve, glad I identified it!??

CBT ?s the only format in which the tesst is offered but if req?ested ?t t??e time of application, certain accommodations in accordance with th? Americans with Disabil?tiies Act (ADA) ?an bee made for specif?c candidates.
After r?aching my acc?untant and ?elay?ng my conce?ns, the next attribute wirth addr?ssing iss communicati?n. There
are some things people f?il to think over and onne of these ?s
de?aling with money.

F?ell free to surf to my blog post; Boca Raton wealth management group

There is nothing I appreciate more compared to visiting this site each life following work. Thanks a lot for every one of the marvelous posts!!!

How do i convert a Joomla design template to basic html/css design template?

N?vertheless, when dealing with this kind of agency,
modest and huge business will be the concentration. To better understand w?? t?ese statistics are so
al?rming one should be famili?r w?th the three ?ategories of fraud; M?nagement Fraud, Employee Fraud, and External Fraud.
By ?ecom?ng p?rt of an accounting team, you will have t?? opportunity too get a?quainted ?ith hhow small businesses operate and
also a?quire practical knowledge about administrative accounting procedures and
federal and st?tutory req?irements.

Herre is my site wealth management boca raton fl

The very heart of your writing while appearing agreeable initially, did not sit properly with me after some time. Someplace throughout the sentences you were able to make me a believer but just for a very short while. I however have got a problem with your jumps in logic and one might do well to help fill in all those breaks. In the event that you actually can accomplish that, I will undoubtedly be amazed.

A large percentage of of what you say is supprisingly accurate and it makes me wonder why I had not looked at this with this light before. This article really did switch the light on for me as far as this particular subject matter goes. Nevertheless at this time there is one factor I am not too comfortable with so whilst I make an effort to reconcile that with the actual core idea of your issue, allow me observe exactly what the rest of the visitors have to point out.Nicely done.

I really enjoy studying on this internet site, it has got great posts. “Violence commands both literature and life, and violence is always crude and distorted.” by Ellen Glasgow.

Hello. I have not been on this website since June. It was because I was visiting to Washington because of my job at Capital One Financial Corp. This takes a lot of time from me, but I still think about your content. I remember our days at College of Saint Mary (NE) in Florida. So thrilled that we had a experience to have fun near the park. Remember to forward my respect to Nicole along with paul. Tchao

Regards for helping out, superb information.

Amazing! Its genuinely amazing paragraph, I have got much clear idea about from this paragraph.|

In Firefox – How to open in new tab automatically when I click a bookmark?

The heart of your writing whilst sounding agreeable originally, did not settle well with me personally after some time. Someplace within the paragraphs you managed to make me a believer unfortunately only for a short while. I nevertheless have a problem with your jumps in logic and one would do nicely to help fill in those gaps. When you can accomplish that, I will undoubtedly be fascinated.

Along with the whole thing which appears to be building within this particular subject material, all your perspectives tend to be rather radical. Nonetheless, I beg your pardon, but I do not subscribe to your whole theory, all be it refreshing none the less. It looks to everyone that your opinions are actually not totally justified and in fact you are yourself not entirely convinced of your argument. In any event I did take pleasure in reading it.

Greetings. I am a completely new Brattice Builder and was wishing to better be prepared for my project at Archer-Daniels-Midland Company. This is why I came to your website. Being raised in Maxwell A F B and going to school at Louisiana State University–Shreveport (LA) in Gallion, has assisted me to prepare for my future. My father and mother are really also educators and they explained me about this write up after studying Shhh! Little Mouse. Appreciate it

Wonderful story, reckoned we could combine a number of unrelated information, nevertheless seriously worth taking a look, whoa did one particular understand about Mid East has got much more problerms too.

Wow! Thank you! I permanently needed to write on my blog something like that. Can I include a fragment of your post to my website?

Check beneath, are some absolutely unrelated web-sites to ours, having said that, they are most trustworthy sources that we use.

Leave a Reply