[Beowulf] Big storage

[Jakob Oestergaard]

Hi all,

While not a typical beowulf, I thought the list would have some input for this
"storage cluster" or whatever we should call it :)

I'm looking at getting some big storage. Of all the parameters, getting as low
dollars/(month*GB) is by far the most important. The price of acquiring and
maintaining the storage solution is the number one concern.

The setup will probably have a number of "head nodes" which receive a large
amount of data over standard gigabit from a large amount of remote sources.
Data is read infrequently from the head nodes by remote systems. The primary
load on the system will be data writes.

The head nodes need not see the same unified storage; so I am not required to
have one big shared filesystem. If beneficial, each of the head nodes could
have their own local storage.

The storage pool will start out at around 100TiB and will grow to ~1PiB within
a year or two (too early to tell). It would be nice to use as few racks as
possible, and as little power as possible  :)

It *might* be possible to offload older files to tape; does anyone have
experience with HSM on Linux?  Does it work?  Could it be worthwhile to
investigate?

One setup I was looking at, is simply using SunFire X4500 systems (you can put
48 standard 3.5" SATA drives in each 4U system). Assuming I can buy them with
1T SATA drives shortly, I could start out with 3 systems (12U) and grow the
entire setup to 1P with 22 systems in little over two full racks.

Any better ideas?  Is there a way to get this more dense without paying an arm
and a leg?  Has anyone tried something like this with HSM?

Thanks all, your input will be greatly appreciated! :)

-- 

 / jakob