[Beowulf] recommendation on crash cart for a cluster room: fullcluster KVM is not an option I suppose?

[Gerry Creager]

Don Holmgren wrote:
> 
> On Tue, 13 Oct 2009, Prentice Bisbal wrote:
> 
>> Mark Hahn wrote:
>>>> Even with IPMI, you still need a crash cart of some type to initially
>>>> set up IPMI in the system's BIOS. At the minimum, you need to set 
>>>> the IP
>>>> address that the IMPI interface will listen on (if it's a shared NIC
>>>
>>> afaik, not really.  here's what I prefer: cluster nodes normally come
>>> out of the box with BIOS configured to try booting over the net before
>>> local HD.
>>> sometimes this is conditional on the local HD having no active 
>>> partition.
>>>
>>> great: so they boot from a special PXE image I set up as a catchall.
>>> (dhcpd lets you define a catchall for any not nodes which lack a 
>>> their own
>>> MAC-specific stanza.)  when nodes are in that state, I like to
>>> auto-configure
>>> the cluster's knowlege of them: collect MAC, add to dhcpd.conf, etc.  at
>>> this stage, you can also use local (open) ipmi on the node itself to
>>> configure the IPMI LAN interface:
>>>     ipmitool lan 2 set password pa55word
>>>     ipmitool lan 2 set defgw ipaddr 10.10.10.254
>>>     ipmitool lan 2 set ipsrc dhcp
>>>
>>> none of this precludes tricks like frobing the switch to find the 
>>> port-MAC
>>> mappings of course - the point is simply that if you let unconfigured 
>>> nodes
>>> autoboot into a useful image, that image can help you automate more 
>>> of the
>>> config.
>>
>> My cluster nodes' IPMI share their physical port with the primary NIC.
>> Before using IPMI, I had to enable it in the BIOS and then assign it an
>> IP address in the BIOS, too. I didn't think of using ipmitool. I wonder
>> if I could do all that using ipmitool, without enabling IPMI in the BIOS
>> first.  Anyone know for sure?
>>
>> -- 
>> Prentice
> 
> 
> With all (and various) flavors of IPMI and BMC hardware on Intel, 
> Supermicro, and Asus systems since about 2004, we've been able to use 
> ipmitool or equivalent software in Linux to setup IPMI LAN and other 
> parameters.
> 
> Consistently, though, for systems with serial over LAN, we've always had 
> to configure serial port redirect settings in the BIOS.  The latter is 
> why we've always tried to get vendors to provide a mechanism for 
> replicating BIOS settings from machine to machine without using a crash 
> cart (not always successfully, unfortunately).


We've been working with one vendor to get IPMI working better and more 
predictably with DHCP. Manual config with a crash cart has been a bit 
problemmatical for us, too.  SuperMicro has been real good about custom 
BIOS mods to help us out in that regard.

gerry