[Beowulf] Please help to setup Beowulf

Mark Hahn hahn at mcmaster.ca
Tue Feb 17 22:32:53 PST 2009


> searches. Without array task scheduling this would require 500,000 individual 
> job submissions. The fact that I never met a serious PBS shop that had not

what's wrong with 500k job submissions?  to me, the existence of "array jobs"
is an admission that the job/queueing system is inefficient.  if you're 
saying that the issue is not per-job overhead of submission, but rather 
that jobs are too short, well, I think that's a user problem.  I think it's 
entirely reasonable to require user jobs to consume some minimum cpu time
(say, few minutes).

> - Policy and resource allocation features are very important to people 
> deploying these systems

so I'm curious what that means.  things like "dept A needs to be guaranteed
N cpus, but dept B gets to use whatever is left over"?  or node choice 
based on amount of free disk?  I don't really see why these sorts of issues
would be less important to more parallel environments.

> - Storage speed is often more important than network speed or latency in many 
> cases

which makes me wonder: do bio types consider using map-reduce-like
frameworks?  that is, basically distributing the work to the data.



More information about the Beowulf mailing list