[Beowulf] Small files

[Bernd Schubert]

Hello Tom,

this is rather self advertising, but you might want to try out 
FhGFS/BeeGFS. We spent lots of development resources to have good 
handling of zillions of small files.

On 06/11/2014 09:03 PM, Tom Harvill wrote:
>
> Hello,
>
> This is my first time posting to this list, thanks in advance for any
> time you spend
> replying.
>
> We've found that a large majority of our files (~40MM of ~50MM) are less
> than 10KB.
> We believe our filesystem (lustre) is bottlenecked with IOPs and locking
> related to

It doesn't even need to be locking, just disk seeking for directory 
entries and inodes might be the bottleneck. Try to monitor your OSTs 
with 'iostat -x'. If you see a high "%util" field and and around 8 
blocks (512 byte) in the avgrq-sz field your disk head is probably 
seeking around most of the time (avgrq-sz field gets mixed up with other 
IOs, so you would need to use blktrace to the for the number of meta 
requests, but blktrace output is not that easy to parse...).

If you only have a single meta server that also might be the bottleneck, 
of course.



Cheers,
Bernd