[Beowulf] Anybody using Redhat HPC Solution in their Beowulf

[Ellis H. Wilson III]

On 10/22/10 18:26, Joshua Baker-LePain wrote:
> On Fri, 22 Oct 2010 at 9:30am, Hearns, John wrote
>
>>> Hello List,
>>> My University is going for a new HPC System. I was using Rocks +
>> CentOS
>>> until now but someone suggested to use Redhat HPC Solution with the
>> new
>>> system.
>>
>> Please, please don't "roll your own" system (pun intended).
>> There are lots of companies out there who will provide you with a high
>> quality, supported
>> Beowulf cluster. Heck - some of them are respected controbutors to this
>> list!

I don't think you could find a statement more orthogonal to the spirit 
of the Beowulf list than, "Please, please don't "roll your own" 
system..."  Isn't Beowulfery about the drawing together of inexpensive 
components in an intelligent fashion suited just for your particular 
application while using standardized (and thereby cheap by the law of 
scale) hardware?  I'm not suggesting Richard build his own NIC - but 
there is nothing wrong with using even a distribution of Linux not 
intended for HPC (so long as you're smart about it) and picking and 
choosing the software (queuing managers, tracers, etc) he finds works best.

Also, I would argue if a company is selling you an HPC solution, it's 
either:
1. A true Beowulf in terms of using COTS hardware, in which case you are 
likely getting less than your money is worth or
2. Isn't a true Beowulf because it is using some custom hardware, 
firmware or software (stack or applications or both) that enables it to 
perform a specific task more efficiently than a simpler (COTS) solution 
would.  In this case you'll be getting your monies worth, but you aren't 
buy a Beowulf - you're buying an HPC solution, big/medium/small iron, or 
some other terminology to represent HPC-in-a-box.

There are certainly advantages (as Joshua mentions below) in getting 
HPC-in-a-box.  But they aren't Beowulfs and they rarely are cheap.

> IMO, in the right set of circumstances there's absolutely nothing wrong
> with rolling your own cluster. For example, say you aleady have the
> expertise in house or are budgeting a FTE into the cluster costs (not to
> mention "free" undergrad or grad student labor). In those cases, why pay
> extra for integration you can do yourself? There's certainly a market
> for cluster vendors, but there's also a place for DIY.

Agreed (though as a grad student, I simultaneously shiver and pat myself 
on the back for such cheap labor).  I could be walking a server rack 
onto a limb here, but I personally don't think the "market for cluster 
vendors" is (or possibly more accurately, "should be," though it would 
make me sad) the Beowulf list.  Depending on how large the HPC system is 
that you are considering, the costs involved in "high quality support," 
is often not worth it for nodes in the hundreds.  A reasonably skilled 
sysadmin can setup and handle a couple hundred nodes without too much 
trouble if he's crafty enough to collect for him or herself the right 
scripts and applications to make administering it reasonably painless. 
Plus, if a chunk of hardware goes (since it's COTS) it's painless to get 
a new one online and put it in yourself when you get in.

Now if you organization wants nodes in the thousands, tens of thousands, 
etc. then you probably don't want a "Beowulf" anyhow.

Best,

ellis