[Beowulf] Some clarification

[Michael Will]

William Hopkins wrote:

>Hello! I can't seem to find the kind of clarification
>I want on Beowulf
>clusters, so I'll ask here:
>
>To my understanding, a beowulf cluster is a cluster
>designed for
>processing power as opposed to redundancy, 
>
mostly agreed on.

>comprised
>of a bunch of
>stripped-down workstations 
>
Can be stripped down workstations or brand-new racked 1U systems that
are diskless and boot via PXE off of the headnode, use infiniband for
faster message passing etc.

>running a software that
>accepts messages
>remotely, so you can write an application that uses
>the remote systems'
>processors as if it were a simple multi-processor
>environment? It
>doesn't yield any extra power to existing, non
>beowulf-optimized
>applications?
>  
>
Correct. Except that you could run several of those in parallel with 
different
input parameters. So in particular if you have an application like image 
processing,
and a bunch of images to process, you could script some batching around it
to run as many in parallel as you have available nodes.

In most beowulf installations (like Scyld) you could just throw all the 
individual
jobs into the queue and it would be processed with one job on one CPU until
all jobs are done.

So it won't help to give more power to a single instance of a serial 
program but
it makes it easy to run multiple copies in parallel.

>Or is there a way to transparently use the remote
>processors as if local, thus allowing smart software
>like 3d Studio MAX to recognize and automatically use
>the extra processing power? Of course with 3dsmax I
>could also install the remote client to each network
>machine, it supports that, but not all softwares do.
>  
>
For beowulf, the answer is usually no. Check out OpenMosix which 
promises a lot
of those things for multithreaded applications, but I would not expect 
it to work
very well in most cases.

Typical problems are: remote ressources are being accessed through the 
network,
and communication overhead can slow down everything worse than necessary
if the transparent automatisms make the wrong decisions for you.

Michael Will

-- 
Michael Will
Penguin Computing Corp.
Sales Engineer
415-954-2822
415-954-2899 fx
mwill at penguincomputing.com