[Beowulf] best linux distribution

[Tim Cutts]

On 8 Oct 2007, at 4:21 pm, Mark Hahn wrote:

> the distribution has nothing to do with your hardware.
> just choose a distro that you are comfortable with - there cannot  
> possibly be any general answer, since all extremes of personal/ 
> professional preference are represented.
>
> personally, I choose RH-ish distros (centos, fedora) mainly because
> they are fairly conventional in structure as visible to the admin.
> for instance, no gratuitous reinvention of sysvinit, widely used  
> package format, etc.
>
> IMO, your master node is over-powered both cpu and memory-wise.
> if you have a single master node in a cluster, it performs three  
> main duties:
> 	- fileserver.  this requires practically no cpu or memory, just disks
> 	and net.  if you have a cachable read-heavy workload, then more
> 	memory on a fileserver may not be wasted.
> 	- cluster management, such as syslogs, scheduler, etc.  very low
> 	cpu or memory load here.

That's not always true.  It depends on the scheduler and the  
workload.  LSF, for example, keeps its list of pending jobs in RAM as  
well as on disk.  This can make the scheduler use a lot of memory if  
the number of pending jobs is large (and for embarrassingly parallel  
workloads that number can be very large indeed).  Our scheduler node  
(with 8GB RAM) ran out of memory once when a particularly over- 
zealous user submitted 1.5 million jobs in one go...  needless to  
say, the user in question had the cluebat vigorously applied, but to  
LSF's credit, it kept going.  Just veeerrrrryyyy  
sllllooooowwwwlllllyyyyy.

> 	- logins.  this might be the place people run the compiler, etc.
> 	but unless they're doing heavy visualization, only a small fraction
> 	of a cpu per user is necessary, and not much memory.
>
> rather than spending a lot of money on a highly configured master  
> node,
> I'd probably split admin/fileservice to a box "uncontaminated" by  
> users.

But still, I agree with you, in general.  Actually, these days, we  
keep the LSF master on a separate node from the login nodes, to  
protect it from wayward user action.

Tim



-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.