[Beowulf] High performance storage with GbE?
cousins at umeoce.maine.edu
Tue Dec 12 10:58:20 PST 2006
We are currently looking to upgrade storage on a 256 node cluster for both
performance and size (around 100 TB). The current thread on distributed
file storage talks about a number of issues we are dealing with.
We are trying to spec out storage that would give us somewhere in the
neighborhood of 5 to 10 MB/sec at each node which translates to around 1.2
to 2.5 GB/sec when it gets to the storage. Of course, this is only if all
nodes are trying to access/write data at once but for some models this is
a possibility. We are trying to do this with GbE since we can't afford to
add FC type cards to each node.
Some vendors use NAS gateways to consolidate a number of FC DAS RAID
arrays together and then serve out NFS via multiple GbE pipes. Others
(Panasas, Isilon) do distributed storage that scales up as you add storage
shelves. Then there are distributed file systems like PVFS and Lustre
that would allow us to use node disks as scratch space. Since we are on a
fairly tight budget I don't think we can afford to do all Panasas/Isilon
type storage. We are probably going to need to do a tiered system with
faster and slower storage with scratch and permanent space. Although a
tape silo is a possibility for part of the storage, for now we want to go
with all disk storage.
My question is: what are other people doing to be able to provide fast
GbE storage to the nodes?
Steve Cousins, Ocean Modeling Group Email: cousins at umit.maine.edu
Marine Sciences, 452 Aubert Hall http://rocky.umeoce.maine.edu
Univ. of Maine, Orono, ME 04469 Phone: (207) 581-4302
More information about the Beowulf