[Beowulf] Putting /home on Lusture of GPFS

[Prentice Bisbal]

On 12/23/2014 09:09 PM, Bill Wichser wrote:
>> On Tue, Dec 23, 2014 at 12:12 PM, Prentice Bisbal
>> <prentice.bisbal at rutgers.edu> wrote:
>>> I was discussing putting /home and /usr/local for my cluster on a 
>>> GPFS or
>>> Lustre filesystem, in addition to using it just for /scratch.
>
> We too have debated this.  Seems a waste to add some 8 or 20 T to a 
> local cluster when we have this nice, central filesystem available. 
> And it's not like the users aren't already using it for everything now 
> anyway.  Yet we always come back to locality of data.  Or at least 
> locality of the login directory.  The three filesystems versus one 
> filesystem, well, it seems attractive.  Not only to admins but to 
> users as well.
>
> In the end, while we'd like to consolidate everything in one place, 
> there are reasons not to do so.  I suppose the strongest is that 
> clusters are dynamic whereas that central storage, not so much. There 
> were Sandybridge executables, then Westmere, then Ivy. Haswell is just 
> beginning.  So a new cluster with a local home is a great place for 
> these execs, keeping architecture codes distinct. If for no other 
> reason this has perpetuated the current method of keeping these local 
> /home directories.
>
> I have other reasons as well.  But that is perhaps my strongest this 
> month.
>

In that case, I can't wait until January! Only 8 more days!

I see the logic in having separate /usr/local for every cluster so you 
can install optimized binaries for each processor, but do you find your 
users take the time to recompile their own codes for each processor 
type, or did you come up with this arrangement to force them to do so?

Prentice