[Beowulf] RAID for home beowulf

[Tony Travis]

Tomislav Maric wrote:
> Hi everyone,
> 
> I've finally gathered all the hardware I need for my home beowulf. I'm
> thinking of setting up RAID 5 for the /home partition (that's where my
> simulation data will be and RAID 1 for the system / partitions without
> the /boot.
> 
> 1) Does this sound reasonable?

Hello, Tomislav.

So far so good, but RAID5 trades capacity for perfomance...

> 2) I want to put the /home at the beginning of the disks go get faster
> write/seek speeds, if the partitions are the same, software RAID doesn't
> care where they are?

Actually, it does - Read about the 'stride' of an ext3 filesystem:

   http://wiki.centos.org/HowTos/Disk_Optimization

You also need to be aware that RAID5 is not so good when writing to the 
disk, because parity has to be calculated and written to the disk. In 
fact this performance penalty has lead to a campaign against RAID5:

   http://www.baarf.com/

> 3) I'll leave the /boot partition on one of the 3 disks and it will NOT
> be included in the RAID array, is this ok?

I think you'd be better off putting your system one one of your three 
disks, and making a RAID1 for /home from the other two. This will give 
you a perfomance gain because RAID1 writes do not involve generating 
parity, and you will decouple disk access between 'system' and /home. 
You can backup your system disk to the RAID1, or reinstall if it fails.

> 4) I've read about setting up parallel swaping via priority given to
> swap partitions in fstab, but also how it would be ok to create RAID 1
> array of swap partitions for the HA of the cluster. What should I choose?

You're going to have to decide between perfomance and HA: Sorry, you 
can't have both with only three disks. I've built systems with four SATA 
disks each with five partitions: /boot on ext2, swap on RAID1, / and 
/home on RAID5, /backup on RAID5 for much the same reasons you are 
considering doing it too. It works, and you do get HA, but performance 
is not good. I'm now simplifying and upgrading the systems by fitting a 
3ware 8006-2 hardware RAID1 controller and two extra disks for /, swap 
and /backups. I'm using the original four disks with a single partition 
now as a software RAID5 for an ext3 /home filesystem using appropriate 
'stride' and directory indexing.

> I've gone through all the software raid how-tos, FAQs and similar, but
> they are not quite new (date at least 3 years) and there's no reference
> to clusters. Any pointers regarding this?

One thing you need to bear in mind in relation to HA is that software 
RAID does not support hot-swap - That's why I chose the 3ware 8006-2, 
which is not very expensive. It doesn't automatically detect new disks 
and rebuild the RAID, but it does support hot-swap and has a web GUI:

   http://www.3ware.com/products/serial_ata8000.asp

Bye,

   Tony.
-- 
Dr. A.J.Travis, University of Aberdeen, Rowett Institute of Nutrition
and Health, Greenburn Road, Bucksburn, Aberdeen AB21 9SB, Scotland, UK
tel +44(0)1224 712751, fax +44(0)1224 716687, http://www.rowett.ac.uk
mailto:a.travis at abdn.ac.uk, http://bioinformatics.rri.sari.ac.uk/~ajt