[Beowulf] Setting up a new Beowulf cluster

[Geoff Jacobs]

Berkley Starks wrote:
> Thank you all so much for the advice so far.  This has helped me see a
> few more of the things that I did not realize at first.
> 
> For a little info on the project, I developed this project as a tool to
> work on my Senior Thesis in a year or so.  Doing computational nuclear
> physics requires such resources.  It will also be used heavily for Monte
> Carlo Simulations and just about any other form of computational
> physics.  The two named are definite projects that are already on the
> line up for when I do get the cluster up and functional.
> 
> I want to be able to make the cluster easily expandable, in that I will
> be starting with only a few machines (about 2-8), but will be acquiring
> more as time goes on.  The university that I am attending surpluses out
> "old" machines every 4 years, and we have set up a program where we can
> get a percentage of the surplus machines for out cluster.
> 
> So, as for size.  Initially it will be a smaller cluster, but will grow
> as time goes on.
> 
> Being new to the Beowulf world, I am just mainly looking for some advice
> as to what distro to use (I would never dream of setting up a cluster on
> windows) and if there were any little tricks that weren't mentioned in
> the setup how to guides.
> 
> Oh, and I would also like to know if there was a way to set up a task
> priority where if I had only only application running it would use all
> the processors on the cluster, but if I had two tasks sent to the
> cluster then it would split the load between them and run both
> simultaneously, but still using a maximum for the needed processors.
> 
> Thanks again so much,
> 
> Berkley

1) Go straight to a gigE switch. There's really no reason in pricing to
not go with gigE, unless you're getting something for free.
2) Surplus hardware will definitely allow you to work through some of
the kinks. I'm guessing the computers will be p4s? Likely with fast
ethernet. You can try your codes on the built in nics and see how things
scale. You might need to upgrade your network.
3) As far as distro is concerned, what are people familiar with at your
site? Do you project any ISV requirements?

-- 
Geoffrey D. Jacobs

To have no errors
  would be life without meaning
  No struggle, no joy