[Beowulf] commercial clusters

Fri Sep 29 08:32:31 PDT 2006

----- Original Message ----- 
From: "Robert G. Brown" <rgb at phy.duke.edu>
To: "Angel Dimitrov" <stormlaboratory at yahoo.com>
Cc: <beowulf at beowulf.org>
Sent: Friday, September 29, 2006 2:03 PM
Subject: Re: [Beowulf] commercial clusters

> On Tue, 26 Sep 2006, Angel Dimitrov wrote:
>
>>
>> Hello,
>>
>> I have some experience of running of numerical weather models on
>> clusters.
>>
>> Is there many clients for processor time? As I saw the biggest
>> supercomputers in the World are very busy! I'm wondering if it's
>> worthwhile to setup a commercial cluster. Intel are planning for new
>> processors - two CPUs each with quad cores. Two such machines will
>> have power like one 50 GHz CPU:-)
>>
>> Any ideas and comments are welcome!
>
> I'm jumping in briefly and late -- it's been tried before unsuccessfully
> as already noted.
>
> The REASON that is tends to be unsuccessful has to do with the nature of
> the beowulf model, though.  In order to succeed you'd need just the
> right mix of clients.  They'd have to have:
>
>   * Infrequent but large computational needs.  Infrequent because if
> they were frequent it will always be cheaper for them to build and run
> their own cluster.  Large because otherwise you don't NEED a cluster.
>
>   * No computational infrastructure to speak of already.  The marginal
> cost of adding a cluster to an EXISTING server room is pretty much the
> cost of the machines, space, power and cooling, and you cannot retail
> these to somebody for what this would cost them in existing facilities
> and make money.  Their economies of scale are the same as yours, but
> they don't have to pay your salary and profit.
>
>   * Sufficient computational expertise to use parallel programs or large
> scale compute clusters in the first place, with the SMALL exception of
> preexisting commercial code.  This requirement is nearly orthogonal to
> the first two, note -- you're now looking for a compute hacker god
> parallel programmer who has big needs, rarely and no server room.

The above is the biggest problem. That's why you need good software that
'hides' the supercomputer. Basically they want with their windows PC start
your program (see attachment) and click somewhere and then run on a big
supercomputer (for the average guy on the street a big supercomputer is
everything that 2 hands cannot lift).

If it was possible to build your own cluster in easy manner and then run for
example a chessprogram at it in a user friendly way,
there would be 100k+ clusters right now of 64 cpu's and more.

Especially now that 1 chip hardly gets faster, we must think in future more 
and more
in clustered manner.

Just doing simple math what i own myself:

1996 : 200Mhz p6
2006 : 2.4Ghz opteron

both are 3 instructions per cycle processors.
So hardware guys won exactly factor 12 in raw processing speed.

Moore's law: each 18 month doubling in transistors.
deduction from that: doubling in speed.

Would mean we are faster now a 106 times  ( 2 ^ 6.66 )
So somewhere a factor 8 is missing.

The average user feels very well that there is no doubling in speed each 18 
months.

So offering solutions to normal users to get their favourite program 
executed faster in an user friendly manner is definitely a big market.

Vincent

>   * No ready access to money to build their own cluster and
> infrastructure from the ground up.  Growth equals power in most of these
> arenas, politically -- if the IT department rents compute facilities,
> they are less important and easier to replace.
>
> The number of potential customers who get through this gauntlet are few,
> and they are more likely to seek help from cluster consultants who make
> the cost of entry even lower -- they'll basically build you a cluster,
> install software on it for you, and run it for you and can almost
> certainly eat your lunch since they are perfectly capable of and happy
> to set up a cluster in THEIR server room for some client if the money is
> right.  There may well be companies in this space, in other words, but
> renting out a cluster is incidental and done per client in such a way
> that the client can always take over ownership and as much of management
> as they like.  They don't DEPEND on this market only for bread and
> butter.
>
> Note that any of these needs ALSO apply in webspace or the ASP
> marketplace, but the difference is that there there are many commercial
> apps and that those apps are used by companies with little to no local
> infrastructure beyond a web drop and a network of e.g. Windows boxes.
> No need for any sort of computational expertise or (really) significant
> compute resources.  And at that, ASP or offsite server setup with an ISP
> tends to be "expensive" compared to anything BUT hiring your own systems
> staff...
>
>    rgb
>
>>
>> Angel
>>
>> __________________________________________________
>> Do You Yahoo!?
>> Tired of spam?  Yahoo! Mail has the best spam protection around
>> http://mail.yahoo.com
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org
>> To change your subscription (digest mode or unsubscribe) visit 
>> http://www.beowulf.org/mailman/listinfo/beowulf
>>
>
> -- 
> Robert G. Brown                        http://www.phy.duke.edu/~rgb/
> Duke University Dept. of Physics, Box 90305
> Durham, N.C. 27708-0305
> Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf
> 