[Beowulf] CephFS

[Joe Landman]

On 04/09/2015 11:24 AM, Tom Harvill wrote:
>
> Hello,
>
> Question: is anyone on this list using CephFS in 'production'?  If so,
> what are you using
> it for (ie. scratch/tmp, archive, homedirs)?  In our setup we use NFS
> shared ZFS for /home,
> Lustre for /work (performance-oriented shared fs), and job-specific tmp
> on the worker nodes
> local disk.
>
> What I really want to know is if anyone is using CephFS (without
> headache?) in a production
> HPC cluster in place of where one might use Lustre?

Not yet ... performance is still somewhat off where it needs to be to 
replace Lustre.  I do think this will be changing rapidly though. 
CephFS is still in the not-quite-fully baked state, but it is evolving 
rapidly.

We'd run some HPC type trading benchmarks on it 2 years ago 
(https://stacresearch.com/news/2013/10/28/stac-report-kdb-31-scalable-informatics-ceph-storage-cloud 
).  Ping me offline if you want more info.

Disclosure:  we are not disinterested observers; having business 
relations with all the FS builders.  So take what I write here with a 
few kg of NaCl.

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
e: landman at scalableinformatics.com
w: http://scalableinformatics.com
t: @scalableinfo
p: +1 734 786 8423 x121
c: +1 734 612 4615