[Beowulf] Suggestions to what DFS to use

[Alex Chekholko]

If you have a preference for Free Software, GlusterFS would work, unless
you have many millions of small files. It would also depend on your
available hardware, as there is not a 1-to-1 correspondence between a
typical GPFS setup and a typical GlusterFS setup. But at least it is free
and easy to try out. The mailing list is active, the software is now mature
( I last used GlusterFS a few years ago) and you can buy support from Red
Hat if you like.

Take a look at the RH whitepapers about typical GlusterFS architecture.

CephFS, on the other hand, is not yet mature enough, IMHO.
On Mon, Feb 13, 2017 at 8:31 AM Justin Y. Shi <shi at temple.edu> wrote:

> Maybe you would consider Scality (http://www.scality.com/) for your
> growth concerns. If you need speed, DDN is faster in rapid data ingestion
> and for extreme HPC data needs.
>
> Justin
>
> On Mon, Feb 13, 2017 at 4:32 AM, Tony Brian Albers <tba at kb.dk> wrote:
>
> On 2017-02-13 09:36, Benson Muite wrote:
> > Hi,
> >
> > Do you have any performance requirements?
> >
> > Benson
> >
> > On 02/13/2017 09:55 AM, Tony Brian Albers wrote:
> >> Hi guys,
> >>
> >> So, we're running a small(as in a small number of nodes(10), not
> >> storage(170TB)) hadoop cluster here. Right now we're on IBM Spectrum
> >> Scale(GPFS) which works fine and has POSIX support. On top of GPFS we
> >> have a GPFS transparency connector so that HDFS uses GPFS.
> >>
> >> Now, if I'd like to replace GPFS with something else, what should I use?
> >> It needs to be a fault-tolerant DFS, with POSIX support(so that users
> >> can move data to and from it with standard tools).
> >>
> >> I've looked at MooseFS which seems to be able to do the trick, but are
> >> there any others that might do?
> >>
> >> TIA
> >>
> >
>
> Well, we're not going to be doing a huge amount of I/O. So performance
> requirements are not high. But ingest needs to be really fast, we're
> talking tens of terabytes here.
>
> /tony
>
> --
> Best regards,
>
> Tony Albers
> Systems administrator, IT-development
> Royal Danish Library, Victor Albecks Vej 1, 8000 Aarhus C, Denmark.
> Tel: +45 2566 2383 / +45 8946 2316
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20170213/1a65ca5d/attachment-0001.html>