[Beowulf] HPC and SAN

[Michael Will]

A lot of times clusters need fast access to terabytes of storage. 

The cheap solution is to have DAS (directly attached storage) added to a separate
fileserver or even just the headnode, serving it via NFS through gigabit ethernet. 

Upside: no special software needed. Downside: bottleneck and single point of
failure in the fileserver. 

The more extreme counter-example is to have a fibre channel card in each compute node
and hook that up to a fibre switch which has fibre-attached storage attached as well. 
Unless you want every compute node have their own separate volume, you then still 
need a cluster filesystem that allows read/write access to the same volume 
from more than one node - the linux filesystems as they come out of the box are not capable 
of doing so. Think about the cached data invisible to the other nodes etc.
Veritas has something called VxFS that could be used for that, and there also special cluster-filesystems
like gfs and lustre that are supposed to solve that problem. In that case, you can also have just some
compute nodes act as storage nodes, and so you don't need fibre channel cards in all of them. The
storage nodes then act similar to redundant nfs servers.

Another interesting case is PVFS (and hopefully soon PVFS2) that accumulates local storage of the
nodes into a parallel virtual filesystem allowing distributed storage and access. In case of PVFS
the data is not distributed redundandly, which means that one node going down means part of
your filesystem data disappears - so unless you have rock solid nodes connected to a UPS, this
might be good only for a large fast /tmp.

Michael Will


On Thursday 16 December 2004 08:52 pm, Rajiv wrote:
> Dear All,
>     Is there any thing like Beowulf cluster and SAN. I would like to have all the data in the Beowulf cluster to be in SAN also. Pls excuse if in case you find my question silly.
> 
> Regards,
> Rajiv

-- 
Michael Will, Linux Sales Engineer
NEWS: We have moved to a larger iceberg :-)
NEWS: 300 California St., San Francisco, CA.
Tel:  415-954-2822  Toll Free:  888-PENGUIN
Fax:  415-954-2899 
www.penguincomputing.com