[Beowulf] commercial clusters

[Robert G. Brown]

On Tue, 26 Sep 2006, Angel Dimitrov wrote:

>
> Hello,
>
> I have some experience of running of numerical weather models on
> clusters.
>
> Is there many clients for processor time? As I saw the biggest
> supercomputers in the World are very busy! I'm wondering if it's
> worthwhile to setup a commercial cluster. Intel are planning for new
> processors - two CPUs each with quad cores. Two such machines will
> have power like one 50 GHz CPU:-)
>
> Any ideas and comments are welcome!

I'm jumping in briefly and late -- it's been tried before unsuccessfully
as already noted.

The REASON that is tends to be unsuccessful has to do with the nature of
the beowulf model, though.  In order to succeed you'd need just the
right mix of clients.  They'd have to have:

   * Infrequent but large computational needs.  Infrequent because if
they were frequent it will always be cheaper for them to build and run
their own cluster.  Large because otherwise you don't NEED a cluster.

   * No computational infrastructure to speak of already.  The marginal
cost of adding a cluster to an EXISTING server room is pretty much the
cost of the machines, space, power and cooling, and you cannot retail
these to somebody for what this would cost them in existing facilities
and make money.  Their economies of scale are the same as yours, but
they don't have to pay your salary and profit.

   * Sufficient computational expertise to use parallel programs or large
scale compute clusters in the first place, with the SMALL exception of
preexisting commercial code.  This requirement is nearly orthogonal to
the first two, note -- you're now looking for a compute hacker god
parallel programmer who has big needs, rarely and no server room.

   * No ready access to money to build their own cluster and
infrastructure from the ground up.  Growth equals power in most of these
arenas, politically -- if the IT department rents compute facilities,
they are less important and easier to replace.

The number of potential customers who get through this gauntlet are few,
and they are more likely to seek help from cluster consultants who make
the cost of entry even lower -- they'll basically build you a cluster,
install software on it for you, and run it for you and can almost
certainly eat your lunch since they are perfectly capable of and happy
to set up a cluster in THEIR server room for some client if the money is
right.  There may well be companies in this space, in other words, but
renting out a cluster is incidental and done per client in such a way
that the client can always take over ownership and as much of management
as they like.  They don't DEPEND on this market only for bread and
butter.

Note that any of these needs ALSO apply in webspace or the ASP
marketplace, but the difference is that there there are many commercial
apps and that those apps are used by companies with little to no local
infrastructure beyond a web drop and a network of e.g. Windows boxes.
No need for any sort of computational expertise or (really) significant
compute resources.  And at that, ASP or offsite server setup with an ISP
tends to be "expensive" compared to anything BUT hiring your own systems
staff...

    rgb

>
> Angel
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu