Comments on: Disposable HPC http://www.linux-mag.com/id/7792/ Open Source, Open Standards Sat, 05 Oct 2013 13:48:18 +0000 hourly 1 http://wordpress.org/?v=3.1 By: stevemadere http://www.linux-mag.com/id/7792/#comment-8375 stevemadere Wed, 30 Nov -0001 00:00:00 +0000 http://www.linux-mag.com/id/7792/#comment-8375 <p>It is amusing to see the same article quote the enormous implied (and at least partially hidden) TCO of a cluster and then advocate using a larger number<br /> of cheaper, less reliable nodes rather than fewer more expensive nodes<br /> with the maximum power possible.</p> <p>Are you assuming that cost of administration rises super-linearly with CPU power?<br /> If not, your solution is actually making the problem worse!</p> <p>Additionally, one has to be very clever to forestall Amdahl\'s law<br /> when deploying slower nodes.</p> <p>I suspect the cost of implementing that cleverness far, far exceeds the savings<br /> from using cheaper nodes.</p> <p>The solution may indeed be disposable nodes but more likely they<br /> are the fastest nodes you can get your hands on and the big<br /> change in mindset is the willingness to treat such an expensive<br /> piece of equipment as disposable. </p> It is amusing to see the same article quote the enormous implied (and at least partially hidden) TCO of a cluster and then advocate using a larger number
of cheaper, less reliable nodes rather than fewer more expensive nodes
with the maximum power possible.

Are you assuming that cost of administration rises super-linearly with CPU power?
If not, your solution is actually making the problem worse!

Additionally, one has to be very clever to forestall Amdahl\’s law
when deploying slower nodes.

I suspect the cost of implementing that cleverness far, far exceeds the savings
from using cheaper nodes.

The solution may indeed be disposable nodes but more likely they
are the fastest nodes you can get your hands on and the big
change in mindset is the willingness to treat such an expensive
piece of equipment as disposable.

]]> By: truly64 http://www.linux-mag.com/id/7792/#comment-8376 truly64 Wed, 30 Nov -0001 00:00:00 +0000 http://www.linux-mag.com/id/7792/#comment-8376 <p>Having increased the size of our linux cluster 5 fold over the last 4 years, I see no 5 fold increase in TCO. The total cost of ownership has been flat. In other words, in my experience, it costs just as much to own/manage a 100 node cluster as a 1000 node cluster.</p> <p>First of all, system managers get very good at adding nodes easily, so each additional node is a negligible extra cost or effort. </p> <p>Secondly, with commodity hardware having a typical 3 year hardware warranty, it makes no sense to put servers onto a costly maintenance contract, as the definition of a cluster means that it can accommodate a node failure by transferring a job to another node. </p> <p>Thirdly, after 3 years, if a node fails, it is removed from the cluster, as it is well past obsolete and is not worth fixing. Three years ago, I was buying dual dualcore servers, now I can get dual 12 core servers for the same price. Why fix the old one?</p> <p>So I find the Myricom analysis differs greatly from that observed in reality. One can create very impressive levels of computing power very cheaply, using open source os and tools, and maintain it on a shoestring without compromising reliability and availability. That is the beauty of a cluster deployment. </p> Having increased the size of our linux cluster 5 fold over the last 4 years, I see no 5 fold increase in TCO. The total cost of ownership has been flat. In other words, in my experience, it costs just as much to own/manage a 100 node cluster as a 1000 node cluster.

First of all, system managers get very good at adding nodes easily, so each additional node is a negligible extra cost or effort.

Secondly, with commodity hardware having a typical 3 year hardware warranty, it makes no sense to put servers onto a costly maintenance contract, as the definition of a cluster means that it can accommodate a node failure by transferring a job to another node.

Thirdly, after 3 years, if a node fails, it is removed from the cluster, as it is well past obsolete and is not worth fixing. Three years ago, I was buying dual dualcore servers, now I can get dual 12 core servers for the same price. Why fix the old one?

So I find the Myricom analysis differs greatly from that observed in reality. One can create very impressive levels of computing power very cheaply, using open source os and tools, and maintain it on a shoestring without compromising reliability and availability. That is the beauty of a cluster deployment.

]]> By: markhahn http://www.linux-mag.com/id/7792/#comment-8377 markhahn Wed, 30 Nov -0001 00:00:00 +0000 http://www.linux-mag.com/id/7792/#comment-8377 <p>the numbers in that paper are fairly bizarre: I can\'t really imagine how it could cost $3m to support 4.1 servers purchased for $160k and serving 4500 users. note that these TCO numbers actually exclude space, power, backup, connectivity, hw/sw support contracts and downtime!</p> <p>the TCO for HPC is much lower - more like 10% per year, even including power. that puts your $400 node in a rather different light - do you mind spending a half-hour per year replacing a failed ATX PSU, etc?</p> <p>not to mention that from a green perspective, \"disposable\" pretty much always means \"unsustainable\" and \"giant carbon footprint\". </p> the numbers in that paper are fairly bizarre: I can\’t really imagine how it could cost $3m to support 4.1 servers purchased for $160k and serving 4500 users. note that these TCO numbers actually exclude space, power, backup, connectivity, hw/sw support contracts and downtime!

the TCO for HPC is much lower – more like 10% per year, even including power. that puts your $400 node in a rather different light – do you mind spending a half-hour per year replacing a failed ATX PSU, etc?

not to mention that from a green perspective, \”disposable\” pretty much always means \”unsustainable\” and \”giant carbon footprint\”.

]]>