cluster setup - handling user homeareas - from main public network storage device

[shin at guss.org.uk]

Hi,

I'm just starting out with my first Beowulf, and have worked out
mostly how I'm going to be setting things up but I'm unsure as to how
to deal with user home areas.

The cluster will probably be configured as a number of nodes on a
class B private address (it's not going to grow too much), with the
front end node (FEN) sitting on the private network and also the
main public network.

The FEN will allow ssh only and will NFS export a number of s/w
packages to the nodes (to save installing s/w on each node). It will
also have a small scratch area - as does each node.

The main problem I'm looking for advice on is how to deal with the
handling of user home areas. All the users have a large storage
allocation on our main RAID (connected to sun/solaris kit, quota'd &
backed up regularly) which sits on the main network, currently users
produce very large (Gb's worth) data files and the smallish area on
the cluster won't suffice - as I expect similarly sized output files
on the cluster.

Should I :

1. automount the users home area from the raid on to the FEN when each
user logs in - but then how do I cope with the fact that the nodes are
on the private network - do I get the FEN to re-export the homearea to
the nodes so that the jobs can write the data back? Or is NAT the
answer somehow here?

2. Setup lots of scratch space (or even use PVFS or similar across all
the nodes local disks) on the FEN which each node can write to and the
users use scp to transfer files to/fro the RAID. Expect users to
balk at the idea of using scp.

Also should I allow users to run jobs (interactive?) on the FEN or
should it be used exclusively for logging in, NFS etc? 

Additionally the main network uses NIS for authentication - and I
wanted to try something similar on the cluster (which will have a
far smaller number of users than the main network) so I was planning
on running a seperate small NIS domain on the cluster (with the FEN
as master), rather than trying to sync passwd etc across nodes.

Any ideas, practical advice/setups on how others are doing/dealing
with user home areas would be appreciated,

Many TIA
Shin