[Beowulf] List traffic

[Vincent Diepeveen]

Well i feel small clusters of say 2 computers or so might get more  
common in future.

Yet let's start asking:

What is a cluster however?

That's not such a simple answer.

Having a few computers at home connected via a router with simple  
default ethernet
is something many have at home.

Is that a cluster?

Maybe.

Let me focus pon the clusters with a decent network.

The decent network clusters suffer from a number of problems.

The biggest problem for this list:

0) yesterday i read in the newspaper another Irani scientist was  
killed by a carbomb.

Past few years i really missed experts posting in here and some dorks  
who really have nothing to contribute to the cluster world,
and just are there to be here, like Jonathan Aquilina, they get back  
in return. So experts leave and idiots come back.

This has completely killed this mailing list.

1) The lack of postings by RGB past few months, especially the ones  
where he explains how easy
it is to build a nuke, given the right ingredients, which gives  
interesting discussions.


Let's look to clusters:

10) the lack of software support for clusters

This is the real big issue.

Sure you can get expensive commercial software to run on clusters,   
but that's all
interesting just for scientists.

Which game can effectively use cluster hardware and is dirt cheap?

This really is a big issue.

Note i intend to contribute myself there to change that, but that's  
just 1 person of course.
Not an entire market moving there

11) the huge break even point of using clusterhardware

I can give examples that i sat here at home with next to me Don  
Dailey, the programmer of Cilkchess,
which used Cilk from Leierson. We played Diep at a single cpu against  
Cilkchess single cpu and Cilkchess
got total toasted.

After having been fried for 4 consecutive games, Don had enough of it  
and disconnected the connection
to the cluster, from which he used 1 cpu for the games, and started  
to play at a version at his laptop,
which did NOT use CILK. So no parallel framework.

It was factor 40 faster.

Now note that at tournaments they showed up with 500 or even 1800 cpu's,
yet you can't have a cluster of 1800 cpu's at home.

Usually building a 4 socket box is far easier, though not necessarily  
cheaper, and practical faster than a small cluster.

Especially AMD has a bunch of cheap 4 socket solutions int he market,  
if you buy those 2nd hand ,there is not really
any competition there from 4 socket clusters in the same price range.

100) the huge increase in power consumption lately of machines. Up to  
2002 i used to visit
     someone, Jan Louwman, who had 36 computres at home, testing  
chessprograms at home.
    So that wasn't a cluster, just a bunch of machiens, in sets of 2  
machines connected with a special
    cable we used to play back then machines against each other.

    Nearly all of those machines was 60-100 watt or so.

    He had divided his computers over 3 rooms or so, majority in 1  
room though. There the 16 ampere @ 230 volt
     power plug already had problems supplying this amount of  
electricity. Around the power plug in the wall,
     the wall and plastic of the powerplug were completely black burned.

    As there was only a single P4 machine amongst the computers,
     only 1 box really consumed a lot of power.

Try to run 36 computers at home nowadays. Most machines are well over  
250 watt,
and the fastest 2 machines i've got here eat 410 respectively 270 watt.

That's excluding the videocard in the 410 watt machine, as it's out  
of it currently (AMD HD 6970),
the box has been setup for gpgpu.

36 machines eat way way too much power.

This is a very simple practical problem that one shouldn't overlook.

It's not realistic that the average joe sets up at his popular gaming  
program a cluster of more
than 2 machines or so.

A 2 machine cluster will never beat a 2 socket machine, except when  
each node also has 2 sockets.

So clustering simple home computers together isn't really useful  
except if you really cluster together half a dozen or more.

Half a dozen machines, using the 250 watt measure and another 25 watt  
for each card and 200 watt for the switch,
it's gonna eat 6 * 275 + 200 = 1850 watt. You really need diehards  
for that.

They are there and more than you and i guess,  but they need SOFTWARE  
that interests them that can use it in a very
  efficient manner, clearly proven to them to be working great and  
easy to install, which refers to point 11.

101) most people like to buy new stuff. new cluster hardware is very  
expensive for more than 2 computers as it needs a switch.
          Second hand it's a lot cheaper, sometimes even dirt cheap,  
yet that's already not what most people like to do

110) Linux had a few setbacks and got less attractive. Say when we  
had redhat end 90s with x-windows it was slowly improving
       a lot. Then x64 was there together with a big dang and we went  
back years and years to x.org.

       X.org threw back linux 10 years in time. It eats massive RAM,  
it's ugly bad, it's slow, it's difficult to configure etc.

      Basically there isn't many good distributions now that are for  
free.

      As most clusters work only very well under linux, the  
difficulty of using linux should really be factored in.

      Have a problem under linux?

      Then forget it as a normal user.

      Now for me linux got MORE attractive as i get hacked total  
silly by every consultant who on this planet knows how to hack on the  
internet,
      yet that's not representative for those with cash who can  
afford a cluster. Note i don't fall into the cash group. My total  
income in 2011 was real little.

111) Usually the big cash to afford a cluster is for people with a  
good job or a tad older, that's usually a different group than the  
group that
         can work with linux. See the previous points for that

Despite all that i believe clusters will get more popular in future,  
for a simple reason: processors don't really clock higher.
So all software that can use additional calculation power already is  
getting parallellized or already has been paralelllized.

It's a matter of time before some of those applications also will  
work well at cluster hardware. Yet this is a slow proces
and it really requires software that works real efficient at small  
number of nodes.

As an example of why i feel this will happen i give to you the  
popularity amongst gamers to run 2 graphics cards connected via a  
bridge with
each other within 1 machine.

Yet the important factor there is that the games really profit from  
doing that.

On Jan 12, 2012, at 4:35 PM, Lux, Jim (337C) wrote:

>
>
> On 1/12/12 6:53 AM, "Ellis H. Wilson III" <ellis at runnersroll.com>  
> wrote:
>>  I
>> recently read a blog that suggested (due to similar threads following
>> these trajectories) that the Wulf list wasn't what it used to be.
>
> I think that's for a variety of reasons..
>
> The cluster world has changed.  Back 15-20 years ago, clusters were  
> new,
> novel, and pretty much roll your own, so there was a lot of traffic  
> on the
> list about how to do that.  Remember all the mobo comparisons, and  
> all the
> carefully teased out idiosyncracies of various switches and network
> schemes.
>
> Back then, the idea of using a cluster for "big computing" was kind of
> new, as well.  People building clusters were doing it either  
> because the
> architecture was interesting OR because they had a computing  
> problem to
> solve, and a cluster was a cheap way to do it, especially with free  
> labor.
>
> I think clustering has evolved, and the concept of a cluster is  
> totally
> mature.  You can buy a cluster essentially off the shelf, from a whole
> variety of companies (some with people who were participating in  
> this list
> back then and still today), and it's interesting how the basic Beowulf
> concept has evolved.
>
> Back in late 90s, it was still largely "commodity computers, commodity
> interconnects" where the focus was on using "business class"  
> computers and
> networking hardware. Perhaps not consumer, as cheap as possible, but
> certainly not fancy, schmancy rack mounted 1U servers.. The switches
> people were using were just ordinary network switches, the same as  
> in the
> wiring closet down the hall.
>
> Over time, though, there has developed a whole industry of supplying
> components specifically aimed at clusters: high speed interconnects,
> computers, etc.   Some of this just follows the IT industry in  
> general..
> There weren't as many "server farms" back in 1995 as there are now.
>
> Maybe it's because the field has matured?
>
>
> So, we're back to talking about "roll-your-own" clusters of one  
> sort or
> another.  I think anyone serious about big cluster computing (>100  
> nodes)
> probably won'd be hanging on this list looking for hints on how to  
> route
> and label their network cables.  There's too many other places to  
> go get
> that information, or, better yet, places to hire someone who  
> already knows.
>
> I know that if I needed massive computational power at work, my first
> thought these days isn't "hey, lets build a cluster", it's "let's  
> call up
> the HPC folks and get an account on one of the existing clusters".
>
> But I still see the need to bring people into the cluster world in  
> some
> way.  I don't know where the cluster vendors find their people, or  
> even
> what sorts of skill sets they're looking for.  Are they beating the  
> bushes
> at CMU, MIT, and other hotbeds of CS looking for prior cluster design
> experience?  I suspect not, just like most of the people JPL hires  
> don't
> have spacecraft experience in school, or anywhere.  You look for  
> bright
> people who might be interested in what you're doing, and they learn  
> the
> details of cluster-wrangling on the job.
>
>
> For myself, I like probing the edges of what you can do with a  
> cluster.
> Big computational problems don't excite me.  I like thinking about  
> things
> like:
>
> 1) What can I use from the body of cluster knowledge to do something
> different.  A distributed cluster is topologically similar to one all
> contained in a single rack, but it's different.  How is it different
> (latency, error rate)? Can I use analysis (particularly from early  
> cluster
> days) to do a better job.
>
> 2) I've always been a fan of *personal* computing (probably from many
> years of negotiating for a piece of some shared resource).  It's  
> tricky
> here, because as soon as you have a decent 8 or 16 node cluster  
> that fits
> under a desk, and have figured out all the hideous complexity of  
> how to
> port some single user application to run on it, someone comes out  
> with a
> single processor box that's just as fast, and a lot easier to use.   
> Back
> in the 80s, I designed, but did not build, a 80286 clone using  
> discrete
> ECL logic, the idea being to make a 100MHz IBM PC-AT that would run
> standard spreadsheet software 20 times faster (a big deal when your  
> huge
> spreadsheet takes hours to recalculate).  However, Moore's law and  
> Intel
> made that idea a losing proposition.
>
> But still, the idea of personal control over my computing resources is
> appealing.  Nobody watching to see "are you effectively using those  
> cpu
> cycles".  No arguing about annual re-adjustment of chargeback rates  
> where
> you take the total system budget and divide it by CPU seconds.   
> Ooops not
> enough people used it, so your CPU costs just quadrupled.
>
> 3) I'm also interested in portable computing (Yes, I have a NEC 8201-
> TRS-80 Model 100 clone, and a TI-59, I did sell the Compaq, but I  
> had one
> of those too,  etc.)  This is another interesting problem space..  
> No big
> computer room with infrastructure.  Here, the fascinating trade is  
> between
> local computer horsepower and cheap long distance datacomm.  At some
> point, it's cheaper/easier to send your data via satellite link  to  
> a big
> computer elsewhere and get the results back.  It's the classic 60s  
> remote
> computing problem revisited once again.
>
>
>>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf