[Beowulf] Suggestions to what DFS to use

[Tony Brian Albers]

On 2017-02-13 17:30, Justin Y. Shi wrote:
> Maybe you would consider Scality (http://www.scality.com/) for your
> growth concerns. If you need speed, DDN is faster in rapid data
> ingestion and for extreme HPC data needs.
>
> Justin
>
> On Mon, Feb 13, 2017 at 4:32 AM, Tony Brian Albers <tba at kb.dk
> <mailto:tba at kb.dk>> wrote:
>
>     On 2017-02-13 09:36, Benson Muite wrote:
>     > Hi,
>     >
>     > Do you have any performance requirements?
>     >
>     > Benson
>     >
>     > On 02/13/2017 09:55 AM, Tony Brian Albers wrote:
>     >> Hi guys,
>     >>
>     >> So, we're running a small(as in a small number of nodes(10), not
>     >> storage(170TB)) hadoop cluster here. Right now we're on IBM Spectrum
>     >> Scale(GPFS) which works fine and has POSIX support. On top of GPFS we
>     >> have a GPFS transparency connector so that HDFS uses GPFS.
>     >>
>     >> Now, if I'd like to replace GPFS with something else, what should I use?
>     >> It needs to be a fault-tolerant DFS, with POSIX support(so that users
>     >> can move data to and from it with standard tools).
>     >>
>     >> I've looked at MooseFS which seems to be able to do the trick, but are
>     >> there any others that might do?
>     >>
>     >> TIA
>     >>
>     >
>
>     Well, we're not going to be doing a huge amount of I/O. So performance
>     requirements are not high. But ingest needs to be really fast, we're
>     talking tens of terabytes here.
>
>     /tony
>
>     --
>     Best regards,
>
>     Tony Albers
>     Systems administrator, IT-development
>     Royal Danish Library, Victor Albecks Vej 1, 8000 Aarhus C, Denmark.
>     Tel: +45 2566 2383 <tel:%2B45%202566%202383> / +45 8946 2316
>     <tel:%2B45%208946%202316>
>     _______________________________________________
>     Beowulf mailing list, Beowulf at beowulf.org
>     <mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
>     To change your subscription (digest mode or unsubscribe) visit
>     http://www.beowulf.org/mailman/listinfo/beowulf
>     <http://www.beowulf.org/mailman/listinfo/beowulf>
>
>

Thanks, I'll take a look at these two. Probably has to be open source 
though.

/tony

-- 
Best regards,

Tony Albers
Systems administrator, IT-development
Royal Danish Library, Victor Albecks Vej 1, 8000 Aarhus C, Denmark.
Tel: +45 2566 2383 / +45 8946 2316