[Beowulf] Suggestions to what DFS to use

Greg Lindahl lindahl at pbm.com
Mon Feb 13 11:00:17 PST 2017


On Mon, Feb 13, 2017 at 07:55:43AM +0000, Tony Brian Albers wrote:
> Hi guys,
> 
> So, we're running a small(as in a small number of nodes(10), not 
> storage(170TB)) hadoop cluster here. Right now we're on IBM Spectrum 
> Scale(GPFS) which works fine and has POSIX support. On top of GPFS we 
> have a GPFS transparency connector so that HDFS uses GPFS.

I don't understand the question. Hadoop comes with HDFS, and HDFS runs
happily on top of shared-nothing, direct-attach storage. Is there
something about your hardware or usage that makes this a non-starter?
If so, that might help folks make better suggestions.

-- greg



More information about the Beowulf mailing list