Rationale behind "Unisys is easier to program"

[Robert G. Brown]

On Thu, 22 Feb 2001, Alan Grimes wrote:

> The person who responded to my statement that Unisys was easier to
> program assumed that I still intended to use MPI/PVM in some fassion on
> the Massively multiprocessing Unisys system. That is not the case... I
> could not even get PVM to function in any way whatsoever even after
> buying a book on it (one of my biggest peves against the system),
> Actually what I mean by this is that I tried to help a BSD user up in
> Canada to get a system up among his personal systems... We couldn't do
> it.

Well, it isn't that difficult under linux, really.  Not to dis BSD or
anything, as it is a fine operating system, but there are reasons that
beowulfery has flourished primarily under linux -- linux is more
chaotic, arguably less stable as a result, but genetically/memetically
richer as a result.  BSD systems are lab rats, healthy, clean,
functional, but perhaps relatively inflexible when presented with a
package that might contain food or rat poison. Linux systems are
(occasionally rabid:-) street rats -- presented with poisoned meat
they'll eat it, hack out or evolve a few internal metabolic adjustments
(at the cost of a few rat-lives), and thrive.  Packaged solutions for
the major linux distributions are hence already (usually) digestible
with little additional effort.  (There.  That should be inflammatory
enough to start a whole thread on Linux vs BSD -- Jihad!  Jihad!  Burn
the infidels!:-)

One can usually install either PVM or MPI directly from RPM's (or
equivalent packages in other packing systems, no Debian or Slackware
flames please:-) sometimes from RPM's that come with the distribution
itself and are "guaranteed" to work out of the box.  The biggest
difficulty one usually has is figuring out how to enable users to run
processes with rsh or ssh without a password across the nodes and
possibly how to set certain key environment variables for them in their
rc files so they are defined on all the nodes.  This has been discussed
on the list numerous times, though, and now that ssh is a universal
feature on systems and supports e.g.  /etc/sshrc (in OpenSSH, anyway --
/etc/environment in the older original ssh) one can easily set these
variables once and for all users there regardless of the shell they
employ.

Both PVM and MPI come with some example code as well that I've never had
much difficulty (aside from the above) getting to run in an hour or so
of experimentation, even the first time I unpacked and built the
libraries by hand (long before the advent of RPMs). I agree that the
example code and packaging could be more tutorial, but I suppose that
the distributers assume (not unreasonably) that they are dealing with
folks with strong programming skills, since one rarely moves on to
parallel code until one is semi-expert in serial code development, and
programmers usually are pretty good at banging against obstacles until
they give up (I certainly have spent the last few weeks banging against
an obstacle of my own devising in my own code -- a deep bug in a complex
routine I wrote my very own self:-).

Finally, there is this list as a resource -- if you have difficulty
getting PVM or MPI to run on a cluster environment, posts to this list
have historically been a very productive way to get unstuck.  The list
archives abound with useful information and suggestions on this,
although I do confess that having a proper searchable archive would be
very helpful here -- I wonder how difficult it would be for the Scyld
folks to integrate a keyword search engine interface with the mailman
archives?

> The key feature of the Unisys system is that it is just an advanced
> multiprocessor architecture. That means I do not need PVM/MPI at all, I
> can just use PThreads and write code saying "run this on all available
> processors". This code could be identical for machines with any number
> of processors and still function correctly depending on how efficient
> the synchronization method is. The User is much happier with this system
> because he can just run a program on one OS without having to worry
> about multiple installations or Net-booting.

Which is all fair enough.  Threads are arguably easier to code to than
message passing programs, although there are certainly counter arguments
-- for one, many significant parallel supercomputing applications
already exist that are coded in e.g. MPI and porting an existing MPI
application is probably even easier than writing your own threaded
version.  Scyld has taken much of the hassle out of creating a cluster
(diskless or not) and indeed the scyld beowulf (in concept, as it is
still in development) presents one with an effective low-maintenance
simulacrum of a single headed system.  I would suspect that because
developers can at long last lay hands on a reasonably stable (or at
least controllably stable) "standard beowulf" software package, it might
now be possible to port one of the network threading packages to it (I
believe that they already exist, but I'm not certain as I've never used
one) and provide the convenience of writing a threaded application on a
true beowulf (possibly at some small cost in performance, possibly not).
Even so, a vendor supported shared memory/threaded architecture is just
great if you can afford it in the first place and it is a cost-effective
solution to your problem in the second place.

> Besides, in the unisys machine you are paying something on the order of
> $2,500 for each processor because it has 4mb of cache...

Which is the rub -- for $2500 I can buy five 1 GHz, 256 MB diskless AMD
nodes pretty easily.  Then you are trading off how well your task
parallelizes and scales with 5 x86-GHz of CPU, smaller caches (but still
fairly decent caching subsystems) and the effective parallelism of five
different access points to local memory against the speed and (bigger)
cache and memory botttleneck of a single CPU.  The single CPU may well
complete its thread faster but can easily lose the price/performance war
by a significant margin for a task that partitions and scales well.

As Walter Ligon is fond of saying (and I'm fond of quoting him:-), the
beowulf architecture isn't, really, about being the fastest possible
parallel solution to fine grained, synchronous problems.  It is about
being the CHEAPEST solution to a whole range of parallel problems, some
(but not all!) of which live out in fine grained, synchronous task
space.  So I would never say that the Unisys system, or a Cray, or an
SP, or any big iron chassis is NEVER going to be right for some
particular client or task.

I WOULD say that sweeping statements like Unisys=threads=good (or
whatever), Beowulf=MPI/PVM=bad is likely to be true for a relatively few
parallel supercomputer consumers when the full cost/benefit equation for
those consumers' needs is taken into account and hence, on average, is
false.  For you it may be true, which is just fine -- so buy the Unisys.
I personally would get far more work done with five cheap nodes than one
expensive node, and don't find the effort required to get work out of
the other four nodes all that difficult or onerous.

Like many things, beowulfery has a learning curve and human costs
associated with it.  If you invest the effort required to master it, you
can reap the benefits of more cycles, often at a lower overall cost.
However, you can also opt to spend more money and get alternative
systems with differing benefit profiles and arguably lower learning
barriers or simpler development/implementation systems.  If development
time is very expensive, these may well be cheaper overall in some
environments.  In others, they aren't.  You are beautifully free to make
personal choices and optimize according to your own perceptions of your
needs and pocketbook.

    rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu