[Beowulf] Young novice with a tight pocket book

[Robert G. Brown]

stuffnstuff writes:

> << HTML content follows >>
> 
> I use my computer for massive rendering jobs regularily and wish there 
> were ways of giving it more speed. I am strongly considering putting 
> togethor a cluster, but I don't know the basic requirements. I have read 
> several guides on building a cluster, but none of them seem to give the 
> software and connection requirements. 
>   
> Until I am corrected, I will be under the impression that I can run a 
> cluster with a master node using XP Home and sub-nodes using Windows 98. 

Allow me to correct you.  You almost certainly need to run linux on the
entire cluster to be able to do anything close to "clustering",
especially on a tightly limited budget.

Linux is both free and far more powerful than Win2000 server
professional super duper version plus office pro, a full set of software
development tools, a full set of graphical and visualization tools,
editors and publishing tools and the wazoo, and then some.  Literally
hundreds of software packages are available, some of them Very Cool
Indeed.  All for free.

Among other things there is at least one rendering program (Persistence
of Vision -- POV) that I've used and which has been parallelized to run
on a cluster.  It is a standard favorite app to show parallel speedup
and parallel scaling at cluster or linux expos.  There are probably more
at this point -- there are people working on pro-scale tools for a
variety of high end tasks in this field but I haven't looked over what's
there recently.

Linux is also a straight line for a student into real IT jobs.  If you
learn to run linux, administer it, secure it, build and run a cluster,
you'll have little difficulty landing a part time job in IT in college
and will have a plethora of opportunities to make money along the way.

> Do I need to buy a new operating system? What do I need to physically 
> connect the computers. If I can conjure up a few old computers that would 
> barely be worth selling, is it worth building a cluster? How do I 
> physically do all this? 

It depends on your budget and your goal.  If your goal is to learn
clustering and to be able to make the most of the resource you have (and
your budget is very low) then by all means build a cluster out of old
boxes at hand.  There are a number of sites on the web that have done
this and posted instructions (some of them pre-blog, but that general
idea).  If you make money with your rendering so that there is a return
on investment to speed things up (or if you have a real budget of $1-2K)
then you should consider building yourself a real mini-cluster.  For
exaxmple, four AMD 64 motherboards connected with an ethernet switch
(maybe even gigE) and running diskless from a server made out of any of
the boxes you already own, maybe spruced up with a hard disk and decent
network card.

Physically a starter cluster is trivial.  Take N "nodes" and 1 "server"
(differentiation is mostly a matter of convenience and function).  Put
linux on the server.  Install server software on it -- NFS and
apache/web and maybe NIS (learning learning learning -- buy a copy of
Nemeth Seabass Snyder etc or learn to use the Linux Howtos at tldp.org).
Learn how to set it up as a dhcp and tftp server.  Put PXE NICs into
your nodes (with or without disks, but with all the memory per node you
can afford, at least 256 MB but ideally 512 MB or more per CPU).  Learn
to boot them and (if disks) kickstart install them automagically.  Or
boot them diskless directly into a cluster configuration.  There are
cluster packages that folks will direct you to on this list (if you
can't find references in the list archives, where they are a FAQ and
answer) that will help you install your cluster with very minimal
learning and sometimes with mailing list support.

Minimally a "cluster" is 2+ linux boxes connected by a suitable LAN,
usually at LEAST TCP/IP ethernet.  Once you have that, lots of folks on
the list will help you find, build, install parallel software and run
applications in parallel.

   rgb

>   
> Sorry for being ill informed... 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20050813/78ba7ee9/attachment.sig>