[Beowulf] Re: SGI and Sun: In Memoriam

Wed Apr 1 11:24:20 PDT 2009

My office started using Suns in 1985; our first system
was a loaner Sun 2/120 (Multibus), and the first system
we bought was a 2/130 (VME). We stuck with Sun hardware
(SPARC-based from the time that such was available)
and SunOS all the way through 2003, at which point we
were simply unable to continue on with that strategy.
There were several reasons for this, some more compelling
than others, and some of them unique to our organization.
Perhaps the most compelling reason was that Sun systems
had become unaffordable for our application. There are
several components to this.

First and foremost is that we run quite a few applications
that are highly demanding of the CPU and the CPU-Memory
channel. Some of the applications will beat the hell out of
memory, with datasizes that will overwhelm any amount of
cache. These sorts of applications do not play well with
others (at least on 20th century Sun architecures), and
thus we had no ability to take advantage of Sun's larger
systems, and instead had always focused on using many
"workstation" devices arrayed into server networks. By 2002
the entry-level systems available from Sun had languished
for some time -- the bulk of their development efforts in
the preceding few years had been focused on the huge data
center web and database servers that were being gobbled
up during the .com bubble. Thus with our cluster approach,
using the latest processors became unaffordable. [1] In
an internal memo I wrote in 2003, I noted that to match
the (SPECrate fp) floating-point performance of a $4500
dual-Xeon system, you'd have to spend about $15,000 on a
Sun Fire 280R; to match the integer performance of the
same dual Xeon you'd have to spend about $44,000 on a
quad-processor, 5U Sun Fire V480.  This was *extremely*
difficult to justify.

Other gripes that had brought us to the brink included:

* Sun support was virtually useless, in part I think
because we didn't use the machines the way they had in
mind. Certainly the prospect of the loss of this "support"
never for a moment made us hesitate to move to a new
vendor. Red Hat isn't all *that* much better, but at
least with the source code one stands a fighting chance
of figuring out where a problem is coming from. Also,
the SunOS patching system was at best arcane, and could
not straightforwardly be integrated with an application
patching system. By comparison to Red Hat's up2date it
was a disaster. Of course at this point we just use yum.

* Sun had never managed to fully integrate open source
software tools. The fact that, in the early 2000s, we
were still having to manually package and distribute the
full GNU software suite and a few dozen other open source
packages on our SunOS systems was absurd. Yes, Sun had
taken to distributing much of this in parallel to the OS,
but the selection was somewhat limited, and versions fell
behind, and and the support was minimal. This was a huge
time sink compared to simply installing the bulk of them
from the OS distribution. Of course there remain a few
packages we need to build ourselves, mostly the latest
versions of a few development and analytical tools for
which the the OS vendor tends to keep more stable, but the
volume of that sort of work is far less on a Linux system.

* For most of the time we had been using Sun systems,
SunOS/SPARC had been the development platform of choice,
industry-wide. Pick a Unix-based software package and
chances were that the development platform, and hence
the first port, was SunOS. By the early 2000s, this
had changed and this role was increasingly being taken
on by Linux/x86. This did have some implications for the
quality and level of support for SunOS in these packages.

* For as long as we had been using Sun systems, they had
frustrated me with their focus on bringing in new customers
at the expense of the old. A story: Early on (~1986) it
became obvious to me that to keep more than three or four
systems configured in any consistent manner it would be
virtually impossible without some sort of automation, so
a few of us built a some scripts that would autoconfigure
a Sun system (fstab, exports, passwd, group, printers,
etc) from values we stored in some custom yp (NIS)
databases. With a couple of minor rewrites along the way,
this system served us well until just a few years ago, when
we wrote a very similar system with an PostgreSQL-based
back end. This is despite all the changes from SunOS 2.x to
SunOS 5.x; with a little work our system was even able to
be implemented on Linux. But in the intervening decades,
I have watched Sun come out with management system after
management system, each one obsoleting the last.  I always
felt sorry for anyone who actually implemented one of those
things, because I knew that it would not be long before
they would have to dump everything and start over again
with something more complex and proprietary than what
it replaced. As far as I could tell, these systems were
always used as marketing tools, trying to convince a new
generation of customers to buy into Sun technology. Sun
was always hoping for loyalty from their customers, but
they were never particularly loyal *to* their customers.

It may be that in 2009 they have more competitive offerings
than they did in 2003. But having moved on, I've seen
little incentive to go back.

--Bob Drzyzgula

[1] Partly this was also an internal accounting issue. For
us, anything that costs more than $5000 per device has
to be capitalized, and used, over four years -- placing
an upper limit on how much we were willing to spend per
system. In the late 1990's, we dealt with this by building
several dozen systems in house out of Sun's SPARCengine
AXmp (roughly equivalent to the Ultra Enterprise 450)
and commodity parts. But by 2002 or so, Sun had made even
this approach impossible -- there was no way to order any
UltraSPARC III-based parts for less than $5000 per line
item (IIRC this was partly because they were bundling
memory to keep people from using 3rd party modules) --
thus we were facing a future where our next system upgrade,
were it to be based on SPARC systems, would have to be
capitalized, with many direct and indirect consequences.

On 01/04/09 11:33 -0400, Glen Beane wrote:
> 
> On 4/1/09 10:45 AM, "Joe Landman" <landman at scalableinformatics.com> wrote:
> > 
> > 
> > This is a current religious mantra within Sun, that Solaris/Sparc will
> (eventually) kill Linux and x86/x64.  Um.  No.
> 
> Isn't Sun's biggest business x86-64 hardware running Linux?  My cluster is
> all AMD64 Sun gear and they gave us a killer edu discount on it. I've been
> very happy with their X2200 series as a cluster node. Actually our
> datacenter has a ton of Sun gear in it and we have a site support contract.
> I've been to a bunch of Sun HPC consortiums and while some time has been
> dedicated to the Niagara platform at each, I'd say the biggest focus is on
> x86 hardware.
> 
> I guess I may be in the minority on this list, but I'm a pretty big Sun fan
> (and I don't use Solaris/Sparc).
> 
> -- 
> Glen L. Beane
> Software Engineer
> The Jackson Laboratory
> Phone (207) 288-6153
> 
> 
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf