[Beowulf] ZFS for HPC

[Rayson Ho]

On Tue, Nov 26, 2013 at 7:10 AM, Andrew Holway <andrew.holway at gmail.com> wrote:
> Taking a holistic approach: What does ZFS mean for HPC? I am excited
> about on the fly data compression and snapshotting however for most
> scientific datas I suppose dedupe is a bit pointless.

We were looking at Lustre+ZFS last year, and we set up a testbed on a
Backblaze Storage Pods 2.0 (for experimenting with different FS
layouts only, Storage Pods are not designed for performance). We found
the LLNL Sequoia ZFS/Lustre talk presented at Lustre User Group 2012
the most interesting:

http://www.opensfs.org/wp-content/uploads/2011/11/LUG12_ZFS_Lustre_for_Sequoia.pdf

(The benchmark results are pretty old, and you can find more recent
results by googling: LLNL+Sequoia+ZFS+Lustre )


> Are raid controllers a thing of the past?

If you believe that Cloud Computing is the future, then RAID
controllers are a thing of the past because Amazon EC2 and Google
Compute Engine only give you JBOD.

We tried to run Lustre in Amazon EC2 on a hs1.8xlarge instance (24 *
2TB HDDs), and we will repeat the tests again on the new i2.8xlarge
instances (8 * 800 GB SSD) soon - note that we did the test with our
own patched kernel and installed a custom Amazon Kernel Image, and we
plan to redo a test on production software before we publish results.
Luckily Intel released "Cloud Edition for Lustre" a few months ago,
but we are haven't checked to see if supports ZFS or if it is still
ldiskfs based...

https://aws.amazon.com/marketplace/pp/B00GBFUO9A

Rayson

==================================================
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html



>
> how often to bits flip anyway? What impact does this have? Does
> checksumming save our data?
>
> Ta
>
> Andrew
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf