[Beowulf] Re: Beowulf Digest, Vol 18, Issue 19

[Robert G. Brown]

stuffnstuff writes:

> << HTML content follows >>
> Questions that I still have: Will my Netgear router replace the need for a 
> 10/100/1000base? If not, in what ballpark price range would one of those 

If it has e.g. 4 switched ethernet ports (many of them do) yes, if you
build an N<=4 node cluster.

If not, 100BT switches are so cheap they are almost giveaways.
pricewatch.com reports that lowball ethernet switch prices these days
are roughly $2/port (as in $15 for an 8 port switch).  Closer to my own
heart for a home cluster is a netgear 16 port gigabit switch for $200,
or $12.50 per port.  I'm still using a 16 port netgear 100BT switch at
home that cost me -- gee -- $200 when I bought it.  If I wait another
year or two, of course, I can probably upgrade to 16 GbE ports for
$50...;-) Or, an 8 port GbE switch NOW costs only $70.  Something like
this is just ideal for a small or starter cluster -- it should provide a
much better balance between compute and communication for systems you
equip with GbE NICs.  Those, too, have gone from pretty expensive to
dirt cheap and ubiquitous -- most new motherboards will have integrated
GbE adapters (maybe "good" ones, maybe not) and in any event, add-on
card adapters are as low as $12.

There you should utilize some discretion.  Features you will want/need
include GOOD linux support (check here after using google or asking on
e.g. linuxquestions.org) and PXE support, if there are still cards being
made without it.  A more expensive card (e.g. 3com, Intel) is also
likely to outperform a really cheap card in certain ways as well,
although you may not run any sort of application mix that really cares
-- most of them will do OK with an ordinary high bandwidth stream of
data.  A card that can do rdma or onboard tcp is also very cool for
cluster work (and more expensive) -- probably more than you want to mess
with for a starter cluster.

> come in? If I was to install Linux on my main computer, would that mean 
> that I could no longer use games or Microsoft Office or Acid or Photo 
> editing programs that I have aquired? I am sure that they make special 

You can (given enough disk or two disks per machine or a diskless boot
configuration) dual boot all your systems.  I'd SUGGEST not dual booting
your "cluster server" or "head node" since if it isn't up nothing else
can be.  Disks are cheap, in fact BIG disks are cheap.  The head node
need not be your fastest machine -- it can even be a junker, as long as
it is "modern" enough to be able to hold a big disk and 512+ MB of
memory and a gigabit ethernet card (say, less than 5 years old).

With it set up, and running "diskless nodes", you can reboot the nodes
with PXE as cluster nodes, use them for a computation, reboot them from
disk and poof, they're WinXX again.

> ones for Linux, but that doesn't mean I want to throw away what I have. 
> Are NIC cards like networking cards? I am assuming wired. Where would I 

Network Interface Cards (NIC).

Definitely wired.  Wireless is slow and curiously, I've observed that
PVM sometimes fails for certain configurations of wireless
primary/client virtual clusters.  In at least one release -- might be
fixed now.

> buy them? The office that I work in is relatively small (about 14 people) 

Online, best buy, circuit city, wal mart...

> and my school uses only Macs; where else could I aquire old junkies? Do 
> you have an example of a high technology center that hasn't had time to 
> throw away computers in its backroom? If I buy the power strip and base 

Universities, businesses.  Sometimes people on list will send you
junkers if you pay the shipping.

But remember, NEW barebones computers (which is all you need) cost as
little as $250 in a configuration that is adequate to serve as a cluster
node.

Due to several laws of computing (Amdahl's law and Moore's Law) it is
better (cheaper and faster) to buy one new 3 GHz node than to buy 3
used/old 1 GHz nodes.  The "sweet spot" of price/performance is usually
a full year behind the bleeding edge in clock and hardware advances.
Check out pricewatch.  A barebones Athlon 64 3000+ costs maybe $230.
Add gigabit NIC and memory (or get one with gigabit NIC integrated),
call it $300.  Compare this to adding six 800 MHz junkers you get for
"free", but have to spend $50 on to add memory and a decent NIC.  Don't
forget that to leave six computers on for a year will cost roughly $600
in ELECTRICITY (yes, your parents pay it and it is a "hidden cost" but
that doesn't make it less real:-).  To leave one on instead, even a much
faster one, will only cost (roughly) $100.

Add it up, and do some benchmarks, and you conclude that getting one
fast computer is both as fast or faster and LOTS cheaper than getting
six really old computers for free.  A sad but true fact -- hardware
older than 4 years isn't economical to use in clusters for any purpose
but learning -- for >>production<< of any sort, you're better off
throwing it away and getting new because computer speed doubles at
constant cost every 18 months (Moore's Law), because 2 1 GHz computers
are at BEST as fast as 1 2 GHz computer (all things being equal,
Amdahl's law) and because energy costs are NOT negligible if you're
actually paying for them and they scale with the number of nodes/cpus
much more strongly than with the speed of those cpus.

> station and connect N computers to it as well as install Linux on every 
> single node, then what? What am I missing? What do I still need to set up? 
> I have heard discussions about wanting or not wanting the most powerful 
> computer to be the server, but how do I decide which one to use as the 
> master? When I render my images trying to take full advantage of my stats, 

N computers + linux on a switched network = "a functional cluster".
Everything else depends on installing the right software (easily done
with linux and generally free) and writing or compiling parallelized
applications to run on the cluster (sometimes easy, often not).  Here is
where you start really learning and getting into hacking/coding big
time.  If you like this and consider it "fun" you'll find yourself
working until 4 am for three months and developing a strange compulsion
to wear a pocket protector filled with pens and screwdrivers.  You may
develop an addiction to Jolt cola.  If not, well, you can always
disassemble the cluster and use its parts as gaming machines;-).

> I generally end up using 50% of the CPU while consuming 480Mb of Ram 
> leaving about 100Mb free; what is my limiting factor (the slowest member 
> of the relay race)? Can a new processor be purchased alone and placed into 

PROBABLY memory, but this depends very much on the specific software you
are using so don't trust this.  The answer to your question is not
trivial or general and cannot be given by me or anybody far away; you
will have to work hard to answer it yourself in practice once you've
built everything to order.  Don't worry TOO much about this in your
starter cluster design, except to note that new hardware has much faster
memory and 64 bits is GOOD for memory bound applications.  As in think
AMD 64 instead of Celerons or even 32 bit Xeons...

> a partially-aged computer? My computer recommends buying PC2700 when 

No.

> PC3200 is the same price. Would PC3200 not function in my computer? I 

Depends.  Look up your motherboard on the mfrs site, find its technical
spec sheet.  Might function but be no faster, too.  Generally you need
to match motherboard to CPU to memory, and don't count on ever being
able to upgrade them during their lifetime in anything but QUANTITY of
memory.  Sometimes (ONLY sometimes) you can actually reuse cases and
peripherals and just upgrade motherboard/cpu/memory combinations, but
often enough even that won't work well after four years.

To save on case costs, remember to look up some of the clever stuff that
has been done with motherboards mounted directly on shelving, using
big/collective power supplies.  It can look really cool as well --
something about that "naked hardware" look goes well in a hobby cluster,
although it is a bad idea for most real/production/professional
clusters.

> assume that if the computers are old enough that the cluster would still 
> be hopelessly slower than my main computer anyways; what should be the 
> minimum stats to even consider building a cluster? 

At the moment, I'd use 2+ GHz nodes used, any Athlon 64 new, any P4 new.
Try to "optimize" price against performance (using CPU clock as a decent
comparator WITHIN a family but not BETWEEN families -- just think of the
slowest AMD 64 being faster than the fastest P4 you can possibly
afford).  Usually there is a pretty clear break point where you spend
more on a clockspeed increment than you get -- system cost goes up by
20% (or even 100%) for a 10% increase in CPU speed.

Don't try to bleeding edge optimize, and remember, your REAL goal for
your FIRST cluster is to LEARN, not necessarily get a huge speedup for
your apps (although if you do get new hardware, of course the latter
will happen one way or another).  When you've been doing this for a year
or so and have learned to code a real parallel app, and have SEEN
Amdahl's law (generalized into an algebraic scaling form) in operation,
when you've learned a whole lot more about application and micro
benchmarking, then you can build your "Mark II" cluster and actually
pick specific hardware on the basis of how will it fits YOUR problem
mix.

For learning, even old/junk nodes are useful, as long as you don't mind
spending your parents' money running them.  Keep in mind, though, that
you can always bargain -- "Gee, dad, if I put these six free computers
in the basement, it's gonna cost you an extra $50/month for power, but
if you just give me $600 for new computers, it will only cost $10 and in
two years we'll show a family profit!"  He'll be impressed with your
grasp of economics and will (after he picks himself up from ROTFL) tell
you to "get a job" as he staggers from the room wiping his eyes...;-)

    rgb
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20050816/27ff1852/attachment.sig>