[Beowulf] Putting /home on Lusture of GPFS

[Prentice Bisbal]

Beowulfers,

I have limited experience managing parallel filesytems like GPFS or 
Lustre. I was discussing putting /home and /usr/local for my cluster on 
a GPFS or Lustre filesystem, in addition to using it just for /scratch. 
I've never done this before, but it doesn't seem like all that bad an 
idea. My logic for this is the following:

1. Users often try to run programs from in /home, which leads to errors, 
no matter how many times I tell them not to do that. This would make the 
system more user-friendly. I could use quotas/policies to encourage them 
to use 'steer' them to use other filesystems if needed.

2. Having one storage system to manage is much better than 3.

3. Profit?

Anyway, another person in the conversation felt that this would be bad, 
because if someone was running a job that would hammer the fileystem, it 
would make the filesystem unresponsive, and keep other people from 
logging in and doing work. I'm not buying this concern for the following 
reasons:

If a job can hammer your parallel filesystem so that the login nodes 
become unresponsive, you've got bigger problems, because that means 
other jobs can't run on the cluster, and the job hitting the filesystem 
hard has probably slowed down to a crawl, too.

I know there are some concerns  with the stability of parallel 
filesystems, so if someone wants to comment on the dangers of that, too, 
I'm all ears. I think that the relative instability of parallel 
filesystems compared to NFS would be the biggest concern, not performance.

-- 
Prentice Bisbal
Manager of Information Technology
Rutgers Discovery Informatics Institute (RDI2)
Rutgers University
http://rdi2.rutgers.edu