[Beowulf] Big storage

[Bruce Allen]

Hi Loic,

> [...]
>>
>> In a system with 24 x 500 GB disks, I would like to have usable storage of
>> 20 x 500 GB and use the remaining disks for redundancy. What do you
>> recommend?  If I understand correctly I can't boot from ZFS so one or more
>> of the remaining 4 disks might be needed for the OS.
>>
> This is (in my opinion) probably the only real issue with the X4500.
> The system disk(s) must be with the data disks (since there are "only"
> 48 disks slots) and the two bootable disks are on the same controller
> which effectively make this controller a single point of failure (there
> are easy ways to move the second system disk to another controller, but
> you still need a working "first" controller to boot).

Can you boot from a USB device?  You can have an inexpensive RAID-1 USB 
device for the root and OS.

> Although in our experience, controller failures are rare on the X4500
> (one failure in over a year with a few tens of X4500).

Did you lose data with a controller failure?  I assume can you just move 
the 48 disks to another box.

It will take me some time to digest your other comments.  But I made a 
mistake in what I wrote.  I want to have a 48 disk box with 500 GB disks. 
>From this (raw) 24 TB of storage I want to get 20 TB usable (eg, lose no 
more than 8 disks of the 40 for redundancy and the OS).  I mistakenly 
wrote 20/24 disks and 10 TB in my email.  How would you revise your 
recommendations for 20TB of usable storage?

Cheers,
 	Bruce