[Beowulf] Suggestions to what DFS to use

[Joe Landman]

On 02/14/2017 08:02 PM, Bogdan Costescu wrote:
> I can second the recommendation for BeeGFS. We have it in use for ~4
> years with very good results, by now on 3 different FSes. We also run

I'll freely admit to being biased here, but BeeGFS is definitely 
something you should be evaluating/using.  Even for this case, with 
HDFS, there is a connector:

	http://www.beegfs.com/wiki/HadoopConnector

We've had excellent results with BeeGFS on spinning rust:

	https://scalability.org/2014/05/massive-unapologetic-firepower-2tb-write-in-73-seconds/
	
and the same system at the customer site

	https://scalability.org/2014/10/massive-unapologetic-firepower-part-2-the-dashboard/

(look closely at the plot, and note the vertical axes ... I had messed 
up the scale on it, but that is in thousands of MB/s).

as well as with an NVMe unit

	
	https://scalability.org/2016/03/not-even-breaking-a-sweat-10gbs-write-to-single-node-forte-unit-over-100gb-net-realhyperconverged-hpc-storage/

Excellent performance, ease of configuration is what you should expect 
from BeeGFS.

-- 
Joe Landman
e: joe.landman at gmail.com
t: @hpcjoe
c: +1 734 612 4615
w: https://scalability.org