[Beowulf] Torrents for HPC
landman at scalableinformatics.com
Wed Jun 13 06:55:39 PDT 2012
On 06/13/2012 09:40 AM, Bernd Schubert wrote:
> On 06/09/2012 02:06 AM, Bill Broadley wrote:
>> I've built Myrinet, SDR, DDR, and QDR clusters ( no FDR yet), but I
>> still have users whose use cases and budgets still only justify GigE.
>> I've setup a 160TB hadoop cluster is working well, but haven't found
>> justification for the complexity/cost related to lustre. I have high
>> hopes for Ceph, but it seems not quite ready yet. I'd happy to hear
> What about an easy to setup cluster file system such as FhGFS? As one of
> its developers I'm a bit biased of course, but then I'm also familiar
> with Lustre, an I think FhGFS is far more easiy to setup. We also do not
> have the problem to run clients and servers on the same node and so of
> our customers make heavy use of that and use their compute nodes as
> storage servers. That should a provide the same or better throughput as
> your torrent system.
I'd like to chime in and note that we have customers re-implementing
storage with FhGFS.
Ceph will be good. You can build a reasonable system today with xfs as
the backing store. The RADOS device is an excellent basis for building
Generally speaking none of the cluster file systems will solve the
specific problem in the original post, though some of the cluster file
systems (and various implementation details) will make the problem
indicated to be much less of a problem.
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web : http://scalableinformatics.com
phone: +1 734 786 8423 x121
fax : +1 866 888 3112
cell : +1 734 612 4615
More information about the Beowulf