[Beowulf] Suggestions to what DFS to use
Tony Brian Albers
tba at kb.dk
Mon Feb 13 01:32:00 PST 2017
On 2017-02-13 09:36, Benson Muite wrote:
> Do you have any performance requirements?
> On 02/13/2017 09:55 AM, Tony Brian Albers wrote:
>> Hi guys,
>> So, we're running a small(as in a small number of nodes(10), not
>> storage(170TB)) hadoop cluster here. Right now we're on IBM Spectrum
>> Scale(GPFS) which works fine and has POSIX support. On top of GPFS we
>> have a GPFS transparency connector so that HDFS uses GPFS.
>> Now, if I'd like to replace GPFS with something else, what should I use?
>> It needs to be a fault-tolerant DFS, with POSIX support(so that users
>> can move data to and from it with standard tools).
>> I've looked at MooseFS which seems to be able to do the trick, but are
>> there any others that might do?
Well, we're not going to be doing a huge amount of I/O. So performance
requirements are not high. But ingest needs to be really fast, we're
talking tens of terabytes here.
Systems administrator, IT-development
Royal Danish Library, Victor Albecks Vej 1, 8000 Aarhus C, Denmark.
Tel: +45 2566 2383 / +45 8946 2316
More information about the Beowulf