[Beowulf] distributed file storage solution?

Anand Vaidya anandvaidya.ml at gmail.com
Tue Dec 12 20:15:45 PST 2006

On Saturday 09 December 2006 10:36, samit wrote:
> hello list,
> i am looking for a application (most preferably, but not necessarily OSS or
> free software) and that can create distributed, reliable, fault tolerant,
> decentralized, high preformance file server. I've looked at few P2P file
> storing solutions that store multiple copies of file in different server
> for data reliability but i havent found any solution that would help me
> access a file as flexibly like the local file with high preformance. Cross
> platform solution would be even better as i want to aovoid the SMB overhead
> if its just a linux based solution!
> PLEASE RECOMENT ME SOMETHING!? feel free to suggest any solutions that even
> qualifies half of this criteria!

Have you considered G-FARM? http://datafarm.apgrid.org

OpenSource Grid filesystem by the Japanese researchers.

I did a proof-of-concept for my customer a year ago. There was also a talk on 
G-FARM by Osamu Tatebe at SC2006...

GFARM (Grid File System) stores metadata on an LDAP server and is able to 
replicate files on multiple data/file servers. They recommend you "bring" the 
compute job to the host containing (probably) large files rather than staging 
files in and out.


> -bipin

Anand Vaidya

More information about the Beowulf mailing list