[Beowulf] Re: Purdue Supercomputer

[Joe Landman]

Tim Cutts wrote:

>> I think the main issue is that ipmi needs a lan interface, though I 
>> guess there are some examples of it sharing a port with the host.
> 
> There are also the issues that most vendors' IPMI implentations are 
> broken, to at least some extent.  HP seem to be in the process of 
> abandoning IPMI in favour of something else (whose name escapes me)

Well ... I am not sure I agree that most are broken.  Its a standard, 
and some have taken ... ah ... liberties ... with the implementation of 
it.  And the quality of some of the implementations is really really 
bad.  That and the fact that it touches *so many* subsystems at such a 
low level in a system, they you really want to keep the standard/coding 
as simple as absolutely possible.  Minor bugs in it could lead to 
machine shutdowns, difficult to diagnose power issues, ...

We have found as a safety precaution, that including a console 
server/kvm unit and having power control via addressable/switchable PDU 
is a great backup, especially when we are hundreds of km (or simply 
different timezones) from the units.

Paraphrasing Heinlein here, I would rather ascribe this state of affairs 
to incompetence than malfeasance.

> I guess one of the problems with IPMI is that it isn't in an individual 
> hardware vendor's interest to support it properly.   If we really had a 

I disagree with this assertion.  We want all gear we sell, or we work 
with to support this, so that our management bits can handle anything we 
throw at it.  Ours, theirs, it doesn't matter.  One management interface 
  for all, no matter what/whos kit is there.  Anything else makes IT 
*exciting* which is generally the last thing you want IT to be.

> management interface that worked consistently across all hardware 
> vendors, then it really wouldn't matter whose tin we bought (which they 
> won't like) and there also wouldn't be any need to lock you into 

Hmmm.... blades are an invention to try to de-commoditize commodity 
gear.  They provide better lockin than IPMI ever would.  You can't take 
an IBM blade-center blade and stick it in a Dell, HP, or Sun chassis. 
Or manage it with their tools.

> expensive vendor-specific management software like SIM (which they also 
> wouldn't like).
> 
> I know I'm mainly mentioning HP here, but I don't really want to single 
> them out for criticism - all the main vendors do the same thing.  Such 
> are the realities of business.  Management systems are now one of the 

Hmmm....  maybe we aren't a "main" vendor (I think you mean "tier-1" or 
TLA vendor)

> few things the main vendors can use to distinguish their products from 
> those of the competition.

Management atop IPMI is fairly common, though as indicated, some 
implementations are horribly broken.

When you willingly pay more for kit from the TLA vendors (identical in 
most ways with kit from non-TLA vendors), you need some sort of 
proprietary justification for buying it.  The main hardware 
differentiators these days are the packaging and the label on the front. 
  There are other non-hardware differentiators in terms of service and 
support, and in terms of being able to understand your issues and map 
this to needed functionality/software/hardware/configuration.  Very few 
vendors can do these things well in HPC.


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
        http://jackrabbit.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 866 888 3112
cell : +1 734 612 4615