[Beowulf] A 10, 000-node Grid Engine Cluster in Amazon EC2 (was: Weekend Cloud Reading)

[Rayson Ho]

1) We ran a 10,000-node cluster on Amazon EC2 for Grid Engine
scalability testing a few weeks ago:

http://blogs.scalablelogic.com/2012/11/running-10000-node-grid-engine-cluster.html

We received lots of help from the Amazon guys, and we (= Open Grid
Scheduler & Scalable Logic) wouldn't be able to achieve all this
without their help!


2) Douglas - we also work with Gompute, and if you are interested in
updating the article, I can hook you up with the Gompute guys.

A few other additions to the article: Gompute offers a parallel
filesystem (GPFS), and supports Windows (most HPC as a service vendors
mainly support Linux). Also, Gompute uses the pay per use model for
some of the commercial software. They also have the visualization
infrastructure, and offers the Gompute software stack - but I have not
used those 2 before...


3) Lastly, we should also mention StarCluster from MIT. While it is
not backed by any single vendor, StarCluster is used by lots of
companies - for example the BioTeam recommends it, and we also use it
for some of our Grid Engine testing as well! If one just needs a small
to medium cluster, then StarCluster can provision it for you in EC2
very quickly; for example, a 100-node cluster could be installed in
around 10 minutes - and that was using spot instances, which have a
slightly higher start time due to the bidding process.

Rayson

==================================================
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/



On Fri, Oct 26, 2012 at 5:26 PM, Douglas Eadline <deadline at eadline.org> wrote:
> Since the North East Coast (yea, the capitals are there for you Lux)
> will be under some clouds this weekend I thought my
> recent survey of "HPC Cloud" offerings may be of
> interest. (notice the quotes)
>
>    Moving HPC to the Cloud
>      http://hpc.admin-magazine.com/Articles/Moving-HPC-to-the-Cloud
>
> Some cold water to dump on your head maybe found here:
>
>    Will HPC Work In The Cloud?
>      http://clustermonkey.net/Grid/will-hpc-work-in-the-cloud.html
>
>
> --
> Doug
>
> --
> Mailscanner: Clean
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf