[Beowulf] Torrents for HPC

[Joe Landman]

On 06/13/2012 09:40 AM, Bernd Schubert wrote:
> On 06/09/2012 02:06 AM, Bill Broadley wrote:
>>
>> I've built Myrinet, SDR, DDR, and QDR clusters ( no FDR yet), but I
>> still have users whose use cases and budgets still only justify GigE.
>>
>> I've setup a 160TB hadoop cluster is working well, but haven't found
>> justification for the complexity/cost related to lustre.  I have high
>> hopes for Ceph, but it seems not quite ready yet.  I'd happy to hear
>> otherwise.
>>
>
> What about an easy to setup cluster file system such as FhGFS? As one of
> its developers I'm a bit biased of course, but then I'm also familiar
> with Lustre, an I think FhGFS is far more easiy to setup. We also do not
> have the problem to run clients and servers on the same node and so of
> our customers make heavy use of that and use their compute nodes as
> storage servers. That should a provide the same or better throughput as
> your torrent system.

I'd like to chime in and note that we have customers re-implementing 
storage with FhGFS.

Ceph will be good.  You can build a reasonable system today with xfs as 
the backing store.  The RADOS device is an excellent basis for building 
reliable systems.

Generally speaking none of the cluster file systems will solve the 
specific problem in the original post, though some of the cluster file 
systems (and various implementation details) will make the problem 
indicated to be much less of a problem.



-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615