[Beowulf] Putting /home on Lusture of GPFS

[Ellis H. Wilson III]

On 12/23/2014 12:33 PM, Joe Landman wrote:
>> Anyway, another person in the conversation felt that this would be
>> bad, because if someone was running a job that would hammer the
>> fileystem, it would make the filesystem unresponsive, and keep other
>> people from logging in and doing work. I'm not buying this concern for
>> the following
>
> This happens.  I've seen it happen.  Many people have seen this happen.
> It does happen.

And if the same physical storage is underlying both /home and /scratch, 
regardless of what file systems you have running there, you (and your 
users) are going to eventually be stricken with the above.  If the 
filesystem you decide upon provides some form of prioritization 
(QoS/etc) you might be able to get away with a converged storage pool 
with prioritized /home, but I would still be leery of it.

If I were in your shoes, separate /home and /scratch and more 
shock-treatment of stupid users would be my plan.  If your scratch is 
considerably faster than home as it should be, that should be enough 
encouragement for your power users to migrate away from running on 
/home.  You'll always have some limited set of users running basic stuff 
on /home.  Not a big deal unless it's spread over 
dozens/hundreds/thousands of machines.  Those are the users you need to 
convince running on scratch is the "right" way.  The best way to 
convince them is to demonstrate how much faster it is.

And if you have users writing tons and tons of tiny files, please go 
beat them on my behalf.

Best,

ellis