[Beowulf] A couple of interesting comments

[Joe Landman]

Hi Gary

   A first point, before going anywhere else ... you get what you pay 
for ... most of the time.  The vast majority of rack-n-stack vendors do 
what you describe.  They know one thing, any deviation from that leaves 
them blinking with open mouths ... they deliver what they have a level 
of comfort delivering.

Gerry Creager wrote:
> We recently purchased a set of hardware for a cluster from a hardware 
> vendor.  We've encountered a couple of interesting issues with bringing 
> the thing up that I'd like to get group comments on.  Note that the RFP 
> and negotiations specified this system was for a cluster installation, 
> so there would be no misunderstanding...
> 
> 1.  We specified "No OS" in the purchase so that we could install CentOS 
> as our base.  We got a set of systems with a stub OS, and an EULA for 
> the diagnostics embedded on the disk.  After clicking thru the EULA, it 
> tells us we have no OS on the disk, but does not fail to PXE.

I would say it is likely due to the fact that altering their base 
cluster construction model is a problem (e.g. costs them money).

FWIW:  We boot our nodes diskless during testing, and diskful if this is 
the required state of the cluster.  Nothing like actually testing the 
hardware you are going to deliver in the way your customers are going to 
use it.

This said, it seems ... unlikely ... that this was their purpose.

> 
> 2.  BIOS had a couple of interesting defaults, including warn on 
> keyboard error (Keyboard?  Not intentionally.  This is a compute node, 
> and should never require a keyboard.  Ever.)  We also find the BIOS is 
> set to boot from hard disk THEN PXE. But due to item 1, above, we never 
> can fail over to PXE unless we load up a keyboard and monitor, and hit 
> F12 to drop to PXE.

Egad.  We (by hand) reconfigure the bios specifically so that there are 
no issues like this.

> In discussions with our sales rep, I'm told that we'd have had to pay 
> extra to get a real bare hard disk, and that, for a fee, they'd have 

Heh....  if you want nothing what is it you have to pay? :)

> been willing to custom-configure the BIOS. OK, with the BIOS this isn't 
> too unreasonable: They have a standard BIOS for all systems and if you 
> want something special, paying for it's the norm...  But, still, this is 
> a CLUSTER installation we were quoted, not a desktop.

Agreed.

> 
> Also, I'm now told that "almost every customer" ordered their cluster 
> configuration service at several kilobucks per rack.  Since the team I'm 

This is standard, it costs money to rack and stack.  If you don't want 
it, you don't have to get it.

> working with has some degree of experience in configuring and installing 
> hardware and software on computational clusters, now measured in at 
> least 10 separate cluster installations, this seemed like an unnecessary 
> expense.  However, we're finding vendor gotchas that are annoying at the 

Yeah, in this case, it is unnecessary.  If your team has the expertise, 
you don't need to pay for it.

> least, and sometimes cause significant work-around time/effort.

For the smaller companies that do cluster setup/installs, the idea is 
not to mess the customer up.

> 
> Finally, our sales guy yesterday was somewhat baffled as to why we'd 
> ordered without OS, and further why we were using Linux over Windows for 
> HPC.  Not trying to revive the recent rant-fest about Windows HPC 

You do understand how hard (e.g. how much money is flowing from) 
Microsoft is pushing their solution.  Money talks.

> capabilities, can anyone cite real HPC applications generally run on 
> significant clusters (I'll accept Cornell's work, although I remain 
> personally convinced that the bulk of their Windows HPC work has been 
> dedicated to maintaining grant funding rather than doing real work)?
> 
> No, I won't identify the vendor.

:)




-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 734 786 8452
cell : +1 734 612 4615