[Beowulf] Suggestions to what DFS to use

[Tony Brian Albers]

On 2017-02-13 17:54, John Hanks wrote:
> We've had pretty good luck with BeeGFS lately running on SuperMicro
> vanilla hardware with ZFS as the underlying filesystem. It works pretty
> well for the cheap end of the hardware spectrum and BeeGFS is free and
> pretty amazing. It has held up to abuse under a very mixed and heavy
> workload and we can stream large sequential data into it fast enough to
> saturate a QDR IB link, all without any in depth tuning. While we don't
> have redundancy (other than raidz3), BeeGFS can be set up with some
> redundancy between metadata servers and mirroring between
> storage. http://www.beegfs.com/content/
>
> jbh
>
> On Mon, Feb 13, 2017 at 7:40 PM Alex Chekholko <alex.chekholko at gmail.com
> <mailto:alex.chekholko at gmail.com>> wrote:
>
>     If you have a preference for Free Software, GlusterFS would work,
>     unless you have many millions of small files. It would also depend
>     on your available hardware, as there is not a 1-to-1 correspondence
>     between a typical GPFS setup and a typical GlusterFS setup. But at
>     least it is free and easy to try out. The mailing list is active,
>     the software is now mature ( I last used GlusterFS a few years ago)
>     and you can buy support from Red Hat if you like.
>
>     Take a look at the RH whitepapers about typical GlusterFS architecture.
>
>     CephFS, on the other hand, is not yet mature enough, IMHO.
>     On Mon, Feb 13, 2017 at 8:31 AM Justin Y. Shi <shi at temple.edu
>     <mailto:shi at temple.edu>> wrote:
>
>         Maybe you would consider Scality (http://www.scality.com/) for
>         your growth concerns. If you need speed, DDN is faster in rapid
>         data ingestion and for extreme HPC data needs.
>
>         Justin
>
>         On Mon, Feb 13, 2017 at 4:32 AM, Tony Brian Albers <tba at kb.dk
>         <mailto:tba at kb.dk>> wrote:
>
>             On 2017-02-13 09:36, Benson Muite wrote:
>             > Hi,
>             >
>             > Do you have any performance requirements?
>             >
>             > Benson
>             >
>             > On 02/13/2017 09:55 AM, Tony Brian Albers wrote:
>             >> Hi guys,
>             >>
>             >> So, we're running a small(as in a small number of nodes(10), not
>             >> storage(170TB)) hadoop cluster here. Right now we're on IBM Spectrum
>             >> Scale(GPFS) which works fine and has POSIX support. On top of GPFS we
>             >> have a GPFS transparency connector so that HDFS uses GPFS.
>             >>
>             >> Now, if I'd like to replace GPFS with something else, what should I use?
>             >> It needs to be a fault-tolerant DFS, with POSIX support(so that users
>             >> can move data to and from it with standard tools).
>             >>
>             >> I've looked at MooseFS which seems to be able to do the trick, but are
>             >> there any others that might do?
>             >>
>             >> TIA
>             >>
>             >
>
>             Well, we're not going to be doing a huge amount of I/O. So
>             performance
>             requirements are not high. But ingest needs to be really
>             fast, we're
>             talking tens of terabytes here.
>
>             /tony
>
>             --
>             Best regards,
>
>             Tony Albers
>             Systems administrator, IT-development
>             Royal Danish Library, Victor Albecks Vej 1, 8000 Aarhus C,
>             Denmark.
>             Tel: +45 2566 2383 <tel:%2B45%202566%202383> / +45 8946 2316
>             <tel:%2B45%208946%202316>
>             _______________________________________________
>             Beowulf mailing list, Beowulf at beowulf.org
>             <mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
>             To change your subscription (digest mode or unsubscribe)
>             visit http://www.beowulf.org/mailman/listinfo/beowulf
>
>
>         _______________________________________________
>         Beowulf mailing list, Beowulf at beowulf.org
>         <mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
>         To change your subscription (digest mode or unsubscribe) visit
>         http://www.beowulf.org/mailman/listinfo/beowulf
>
>     _______________________________________________
>     Beowulf mailing list, Beowulf at beowulf.org
>     <mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
>     To change your subscription (digest mode or unsubscribe) visit
>     http://www.beowulf.org/mailman/listinfo/beowulf
>
> --
> ‘[A] talent for following the ways of yesterday, is not sufficient to
> improve the world of today.’
>  - King Wu-Ling, ruler of the Zhao state in northern China, 307 BC

BeeGFS sounds very promising. I'll have a look at that too.

/tony

-- 
Best regards,

Tony Albers
Systems administrator, IT-development
Royal Danish Library, Victor Albecks Vej 1, 8000 Aarhus C, Denmark.
Tel: +45 2566 2383 / +45 8946 2316