[Beowulf] Big storage

[Loic Tortay]

According to Joe Landman:
> Bruce Allen wrote:
> > Hi Loic,
> > 
> >> [...]
> >>>
> >>> In a system with 24 x 500 GB disks, I would like to have usable
> >>> storage of
> >>> 20 x 500 GB and use the remaining disks for redundancy. What do you
> >>> recommend?  If I understand correctly I can't boot from ZFS so one or
> >>> more
> >>> of the remaining 4 disks might be needed for the OS.
> >>>
> >> This is (in my opinion) probably the only real issue with the X4500.
> >> The system disk(s) must be with the data disks (since there are "only"
> >> 48 disks slots) and the two bootable disks are on the same controller
> >> which effectively make this controller a single point of failure (there
> >> are easy ways to move the second system disk to another controller, but
> >> you still need a working "first" controller to boot).
> > 
> > Can you boot from a USB device?  You can have an inexpensive RAID-1 USB
> > device for the root and OS.
> 
> FWIW:  Other solutions can boot from USB, Flash, ... and provide the 48
> drive capability.
> 
Sure, and so can the X4500. :-)

But, with only two RAID controllers there is no way your machine will 
survive a controller failure when using RAID-5 or RAID-6 unless you're 
willing to half the available space (or add the optional extra 
controllers).

The density of the X4500 is also slightly better (48 disks in 4U 
instead of 5U).
As of today we have 112 X4500, 112U are almost 3 racks which is quite 
a lot due to our floor space constraints.

Besides, Sun machine have been available for one year and they can 
deliver and maintain theirs reliably wherever we want (specifically 
in Lyon, France).

The main performance figure on your website is quite pointless (sorry) 
since there is no mention of the number of files, file(s) size(s), 
number of threads, block size and amount of bytes written.

I do 1.7 GB/s with random writes on a X4500, so my <whatever> is 
larger (and pointless as well). ;-)



Loïc.
-- 
| Loïc Tortay <tortay at cc.in2p3.fr> -     IN2P3 Computing Centre     |