[Beowulf] ZFS for HPC

Rayson Ho raysonlogin at gmail.com
Mon Dec 23 09:44:04 PST 2013

On Tue, Nov 26, 2013 at 7:10 AM, Andrew Holway <andrew.holway at gmail.com> wrote:
> Taking a holistic approach: What does ZFS mean for HPC? I am excited
> about on the fly data compression and snapshotting however for most
> scientific datas I suppose dedupe is a bit pointless.

We were looking at Lustre+ZFS last year, and we set up a testbed on a
Backblaze Storage Pods 2.0 (for experimenting with different FS
layouts only, Storage Pods are not designed for performance). We found
the LLNL Sequoia ZFS/Lustre talk presented at Lustre User Group 2012
the most interesting:


(The benchmark results are pretty old, and you can find more recent
results by googling: LLNL+Sequoia+ZFS+Lustre )

> Are raid controllers a thing of the past?

If you believe that Cloud Computing is the future, then RAID
controllers are a thing of the past because Amazon EC2 and Google
Compute Engine only give you JBOD.

We tried to run Lustre in Amazon EC2 on a hs1.8xlarge instance (24 *
2TB HDDs), and we will repeat the tests again on the new i2.8xlarge
instances (8 * 800 GB SSD) soon - note that we did the test with our
own patched kernel and installed a custom Amazon Kernel Image, and we
plan to redo a test on production software before we publish results.
Luckily Intel released "Cloud Edition for Lustre" a few months ago,
but we are haven't checked to see if supports ZFS or if it is still
ldiskfs based...



Open Grid Scheduler - The Official Open Source Grid Engine

> how often to bits flip anyway? What impact does this have? Does
> checksumming save our data?
> Ta
> Andrew
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

More information about the Beowulf mailing list