[Beowulf] A 10, 000-node Grid Engine Cluster in Amazon EC2 (was: Weekend Cloud Reading)
chichan2008 at gmail.com
Thu Dec 6 09:53:27 PST 2012
On Wed, Nov 28, 2012 at 12:10 PM, Rayson Ho <raysonlogin at gmail.com> wrote:
> 1) We ran a 10,000-node cluster on Amazon EC2 for Grid Engine
> scalability testing a few weeks ago:
I saw some other large clusters, and most used larger number of cores
and few nodes.
For example, Numerate’s Drug Design Platform Scales to 10,000+ Cores
Using Spot Instances:
> 2) Douglas - we also work with Gompute, and if you are interested in
> updating the article, I can hook you up with the Gompute guys.
> A few other additions to the article: Gompute offers a parallel
> filesystem (GPFS), and supports Windows (most HPC as a service vendors
> mainly support Linux). Also, Gompute uses the pay per use model for
> some of the commercial software. They also have the visualization
> infrastructure, and offers the Gompute software stack - but I have not
> used those 2 before...
And I think they are also using bare-metal instead of VM, so it can
benefit a few types of parallel apps that require low latency.
> 3) Lastly, we should also mention StarCluster from MIT. While it is
> not backed by any single vendor, StarCluster is used by lots of
> companies - for example the BioTeam recommends it, and we also use it
> for some of our Grid Engine testing as well! If one just needs a small
> to medium cluster, then StarCluster can provision it for you in EC2
> very quickly; for example, a 100-node cluster could be installed in
> around 10 minutes - and that was using spot instances, which have a
> slightly higher start time due to the bidding process.
> Open Grid Scheduler - The Official Open Source Grid Engine
> On Fri, Oct 26, 2012 at 5:26 PM, Douglas Eadline <deadline at eadline.org> wrote:
>> Since the North East Coast (yea, the capitals are there for you Lux)
>> will be under some clouds this weekend I thought my
>> recent survey of "HPC Cloud" offerings may be of
>> interest. (notice the quotes)
>> Moving HPC to the Cloud
>> Some cold water to dump on your head maybe found here:
>> Will HPC Work In The Cloud?
>> Mailscanner: Clean
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf