[Beowulf] Small files

[Jeffrey Layton]

Tom,

Without digging into the details too much, can you describe the Lustre
setup? As Bernd alluded to, it might be something in the configuration or
version that is hampering better performance. But then again, you may not
want to upgrade to a newer because of disruption. But maybe there are some
things that can be done to improve Lustre performance.

Thanks!

Jeff


On Thu, Jun 12, 2014 at 7:42 AM, Bernd Schubert <bs_lists at aakef.fastmail.fm>
wrote:

> Hello Tom,
>
> this is rather self advertising, but you might want to try out
> FhGFS/BeeGFS. We spent lots of development resources to have good handling
> of zillions of small files.
>
>
> On 06/11/2014 09:03 PM, Tom Harvill wrote:
>
>>
>> Hello,
>>
>> This is my first time posting to this list, thanks in advance for any
>> time you spend
>> replying.
>>
>> We've found that a large majority of our files (~40MM of ~50MM) are less
>> than 10KB.
>> We believe our filesystem (lustre) is bottlenecked with IOPs and locking
>> related to
>>
>
> It doesn't even need to be locking, just disk seeking for directory
> entries and inodes might be the bottleneck. Try to monitor your OSTs with
> 'iostat -x'. If you see a high "%util" field and and around 8 blocks (512
> byte) in the avgrq-sz field your disk head is probably seeking around most
> of the time (avgrq-sz field gets mixed up with other IOs, so you would need
> to use blktrace to the for the number of meta requests, but blktrace output
> is not that easy to parse...).
>
> If you only have a single meta server that also might be the bottleneck,
> of course.
>
>
>
> Cheers,
> Bernd
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20140612/12b54810/attachment.html>