[Beowulf] A 10, 000-node Grid Engine Cluster in Amazon EC2 (was: Weekend Cloud Reading)

[Chi Chan]

On Wed, Nov 28, 2012 at 12:10 PM, Rayson Ho <raysonlogin at gmail.com> wrote:
> 1) We ran a 10,000-node cluster on Amazon EC2 for Grid Engine
> scalability testing a few weeks ago:
>
> http://blogs.scalablelogic.com/2012/11/running-10000-node-grid-engine-cluster.html

I saw some other large clusters, and most used larger number of cores
and few nodes.

For example, Numerate’s Drug Design Platform Scales to 10,000+ Cores
Using Spot Instances:

http://numerate.com/blog/?p=155


> 2) Douglas - we also work with Gompute, and if you are interested in
> updating the article, I can hook you up with the Gompute guys.
>
> A few other additions to the article: Gompute offers a parallel
> filesystem (GPFS), and supports Windows (most HPC as a service vendors
> mainly support Linux). Also, Gompute uses the pay per use model for
> some of the commercial software. They also have the visualization
> infrastructure, and offers the Gompute software stack - but I have not
> used those 2 before...

And I think they are also using bare-metal instead of VM, so it can
benefit a few types of parallel apps that require low latency.

--Chi


>
>
> 3) Lastly, we should also mention StarCluster from MIT. While it is
> not backed by any single vendor, StarCluster is used by lots of
> companies - for example the BioTeam recommends it, and we also use it
> for some of our Grid Engine testing as well! If one just needs a small
> to medium cluster, then StarCluster can provision it for you in EC2
> very quickly; for example, a 100-node cluster could be installed in
> around 10 minutes - and that was using spot instances, which have a
> slightly higher start time due to the bidding process.
>
> Rayson
>
> ==================================================
> Open Grid Scheduler - The Official Open Source Grid Engine
> http://gridscheduler.sourceforge.net/
>
>
>
> On Fri, Oct 26, 2012 at 5:26 PM, Douglas Eadline <deadline at eadline.org> wrote:
>> Since the North East Coast (yea, the capitals are there for you Lux)
>> will be under some clouds this weekend I thought my
>> recent survey of "HPC Cloud" offerings may be of
>> interest. (notice the quotes)
>>
>>    Moving HPC to the Cloud
>>      http://hpc.admin-magazine.com/Articles/Moving-HPC-to-the-Cloud
>>
>> Some cold water to dump on your head maybe found here:
>>
>>    Will HPC Work In The Cloud?
>>      http://clustermonkey.net/Grid/will-hpc-work-in-the-cloud.html
>>
>>
>> --
>> Doug
>>
>> --
>> Mailscanner: Clean
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf