[Beowulf] KVM to a compute node

[Julien Leduc]

> What kind of solutions do people go to boot up an individual compute 
> node. We will have nodes with video cards and disks, so they can boot 
> up separately. But I am hoping there is a solution that is better than 
> stringing long KVM wires across the server room.

For the cluster we have in our lab, we could not afford an unusable kvm switch for 48 nodes, so we used the serial line, connectivity is cheap with one 1.8 m null modem cable per node and a USB -> 8 rs232 box plugged on a server.
To scale both on space (1.8 meters is short to reach all the compute nodes), and number of nodes, we use a USB 2.0 hub, to grant enough bandwidth for all our boxes and spread those among the nodes.

These USB->serial boxes work fine with linux (ours uses the ftdi_sio kernel module, integrated in kernel.org sources), and those additional ttys are seen as ttyUSBx in /dev.
We use kermit to log on the local console to each machine, and have aliases for the combination of ssh (to the server) and kermit on the right tty.

Our nodes allow us to access the BIOS settings, and then grub, that has serial line support too, permits us to choose our computing distribution from serial (useless thanks to PXE, but fun for demos), and then, we get the kernel messages, and distribution rc scripts output (I even managed to fix a file system from serial line), and then a local console.

This solution is quite cheap (around 200 euros for 8 nodes), compaired to proprietary ones, gives us the ability to check and fix things locally (very usefull with network problems), and do not rely on higher level protocols (often, a management card is only a 1 port terminal server over an internal tty (console redirection is often set in /etc/inittab), that implies a management network (9600 bauds over 100Mb/s connectivity...), with routing, fitering,... and so new security concerns).

More can be found in kernel Documentation (Documentation/serial-console.txt Documentation/usb/usb-serial.txt).

I suggest you to have a try, I can help if needed.

Julien