[Beowulf] Small files

[Kilian Cavalotti]

On Fri, Jun 13, 2014 at 7:03 AM, Ellis H. Wilson III <ellis at cse.psu.edu> wrote:
> a) Fix it transparently with automatic policies/FS's in the back-end. (I
> know of at least one FS that packs small files with metadata transparently
> on SSDs to expedite small file IOPS, but message me off-list for that as I
> start work for that shop soon and don't want to so blatantly advertise).
> There are limits to how much these policies/FS's can fix though.  Bad I/O
> will still be Bad I/O after a point.

Speaking of which, there's this project from Los Alamos, PLFS
(https://github.com/plfs/plfs-core), that tries to aggregate random
I/O from applications into sequential operations that better suit the
underlying parallel filesystem. Never used it myself, but I've seen a
presentation from Brett Kettering at LUG'14 which outlined nice
performance improvements for some use cases.

Cheers,
-- 
Kilian