[Beowulf] Sil 3124 controller, boot from disk, UUID issues

[David Mathog]

Here is a work around for this issue.  

For whatever reason the Sil 3124 controller reported sda7 as
TYPE="silicon_medley_raid_member", but NOT any of the other partitions.
 Taking a wild guess that the controller was reporting the last
partition this way, and not really doing anything to the partition, I
changed the partition table from:

/dev/sda7       109433088  1953523055 

to

/dev/sda7       109433088  1953523055   922044984   83  Linux
/dev/sda8      1953523120  1953525167        1024   83  Linux

wrote that to disk, then

partprobe /dev/sda
blkid

and now we find

/dev/sda8: TYPE="silicon_medley_raid_member" 

with sda7 back to its original UUID!  This fits the hypothesis that the
UUID wasn't overwritten by the controller, the controller just hides
information about the last partition.  My evil plan seems to be working,
the 8th partition now takes the hit, and there is nothing in the small
8th partition, so that is fine.  Of course the file system on sda7 is
now toast, so...

mkfs -t ext4 /dev/sda7
mount -t ext4 /dev/sda7 /home 
(restore contents of sda7 == /home from remote storage)
nedit /etc/fstab
(put in new UUID for sda7, it changed at mkfs)
reboot

and the system mounts all of the partitions fine, including sda7.

% blkid | sort
/dev/sda1: UUID="ce8b6ea5-8bca-45d3-b9f0-a265e611b929" TYPE="ext4" 
/dev/sda5: UUID="eb79e4d0-66a2-473d-a8fa-5c444e38bb87" TYPE="swap" 
/dev/sda6: UUID="80d7830d-53f5-4b69-8b4b-e67ee1d47c9c" TYPE="ext4" 
/dev/sda7: UUID="d3213c8e-3682-4168-b11e-d0b949aee9c9" TYPE="ext4" 
/dev/sda8: TYPE="silicon_medley_raid_member" 
/dev/sdb1: UUID="caf1e9ea-4eae-41f8-98d1-c4e4cbd6b102" TYPE="ext4" 
/dev/sdb5: UUID="485b0c52-e133-4359-88fa-c01d7d4ae3d1" TYPE="swap" 
/dev/sdb6: UUID="2b0ef732-9526-49c1-9cb9-f0b73c86874c" TYPE="ext4" 

where sdb is just a disk, not part of any raid or concatenation set, and
sda is the single member of a concatenation set so that it can boot.

In summary, linux can work through the 3124 controller without the
controller having to be told about the disks, in which case the 3124
doesn't play games with the partition information. Unfortunately the
only way to make a disk on the controller bootable is to tell the
controller about it, in which case the controller reports odd things for
the last partition. So make sure the last partition is small and not
used by the linux system, and this will not get in the way.  (I don't
know how small it can be for this to work, probably smaller than 1MB,
but I didn't experiment further.)

It would have been nice if some of this was documented somewhere on
either Silicon Image or Syba's sites, but if it was, I could not find it.

Regards,

David Mathog
mathog at caltech.edu
Manager, Sequence Analysis Facility, Biology Division, Caltech