[Beowulf] scratch File system for small cluster

[Tim Cutts]

On 25 Sep 2008, at 3:19 pm, Joe Landman wrote:

>  BLAST uses mmap'ed IO.  This has some interesting ...  
> interactions ... with parallel file systems.

It's not *too* bad on Lustre.  We use it in production that way.

>> Are there other recommendations for fast scratch space (it doesn't  
>> have to
>> be a parallel file system, something with less hardware would be  
>> nice)
>
> Pure software:  GlusterFS currently, ceph in the near future.  GFS  
> won't give you very good performance (meta-data shuttling limits  
> what you can do).  You could go Lustre, but then you need to build  
> MDS/ODS setups so this is hybrid.

Lustre still has some interesting performance corners.  Random access  
with small reads is weak, so don't try putting DBM files on it, for  
example.

>
> Pure hardware:  Panasas (awesome kit, but not for the light-of- 
> wallet), DDN, Bluearc (same comments for these as well).

We have seen some scaling/stability issues with BlueArc NFS heads, at  
least on our SAN hardware.  At the scale the OP is suggesting though,  
it'll be fine (and they certainly are fast).

Regards,

Tim


-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.