[Beowulf] recommendation on crash cart for a cluster room: fullcluster KVM is not an option I suppose?

[Don Holmgren]

On Tue, 13 Oct 2009, Prentice Bisbal wrote:

> Mark Hahn wrote:
>>> Even with IPMI, you still need a crash cart of some type to initially
>>> set up IPMI in the system's BIOS. At the minimum, you need to set the IP
>>> address that the IMPI interface will listen on (if it's a shared NIC
>>
>> afaik, not really.  here's what I prefer: cluster nodes normally come
>> out of the box with BIOS configured to try booting over the net before
>> local HD.
>> sometimes this is conditional on the local HD having no active partition.
>>
>> great: so they boot from a special PXE image I set up as a catchall.
>> (dhcpd lets you define a catchall for any not nodes which lack a their own
>> MAC-specific stanza.)  when nodes are in that state, I like to
>> auto-configure
>> the cluster's knowlege of them: collect MAC, add to dhcpd.conf, etc.  at
>> this stage, you can also use local (open) ipmi on the node itself to
>> configure the IPMI LAN interface:
>>     ipmitool lan 2 set password pa55word
>>     ipmitool lan 2 set defgw ipaddr 10.10.10.254
>>     ipmitool lan 2 set ipsrc dhcp
>>
>> none of this precludes tricks like frobing the switch to find the port-MAC
>> mappings of course - the point is simply that if you let unconfigured nodes
>> autoboot into a useful image, that image can help you automate more of the
>> config.
>
> My cluster nodes' IPMI share their physical port with the primary NIC.
> Before using IPMI, I had to enable it in the BIOS and then assign it an
> IP address in the BIOS, too. I didn't think of using ipmitool. I wonder
> if I could do all that using ipmitool, without enabling IPMI in the BIOS
> first.  Anyone know for sure?
>
> --
> Prentice


With all (and various) flavors of IPMI and BMC hardware on Intel, Supermicro, 
and Asus systems since about 2004, we've been able to use ipmitool or equivalent 
software in Linux to setup IPMI LAN and other parameters.

Consistently, though, for systems with serial over LAN, we've always had to 
configure serial port redirect settings in the BIOS.  The latter is why we've 
always tried to get vendors to provide a mechanism for replicating BIOS settings 
from machine to machine without using a crash cart (not always successfully, 
unfortunately).

Don Holmgren
Fermilab