[Beowulf] HPC in the cloud question

[Dimitris Zilaskos]

Hi,

IBM Platform does provide IB for HPC with bare metal and cloudbursting,
among other HPC services on the cloud. Detailed information including
benchmarks can be found at
http://www-03.ibm.com/systems/platformcomputing/products/cloudservice/ .
Note that I work for IBM so I am obviously biased.

Best regards,

Dimitris

On Fri, May 8, 2015 at 2:40 PM, Prentice Bisbal <prentice.bisbal at rutgers.edu
> wrote:

> Mike,
>
> What are the characteristics of your cluster workloads? Are they tightly
> coupled jobs, or are they embarassingly parallel or serial jobs? I find it
> hard to believe that a virtualized, ethernet shared network infrastructure
> can compete with FDR IB for performance on tightly coupled jobs. AWS HPC
> representatives came to my school to give a presentation on their
> offerings, and even they admitted as much.
>
> If your workloads are communication intensive, I'd think harder about
> using the cloud, or find a cloud provider that provides IB for HPC (there
> are a few that do, but I can't remember their names).  If your workloads
> are loosely-coupled jobs or many serial jobs, AWS or similar might be fine.
> AWS does not provide IB, and in fact shares very little information about
> their network architecture, making it had to compare to other offerings
> without actually running benchmarks.
>
> If your users primarily interact with the cluster through command-line
> logins, using the cloud shouldn't be noticeably different the hostname(s)
> they have to SSH to will be different, and moving data in an out might be
> different, but compiling and submitting jobs should be the same if you make
> the same tools available in the cloud that you have on your local clusters.
>
> Prentice
>
>
>
>
> On 05/07/2015 06:28 PM, Hutcheson, Mike wrote:
>
>> Hi.  We are working on refreshing the centralized HPC cluster resources
>> that our university researchers use.  I have been asked by our
>> administration to look into HPC in the cloud offerings as a possibility to
>> purchasing or running a cluster on-site.
>>
>> We currently run a 173-node, CentOS-based cluster with ~120TB (soon to
>> increase to 300+TB) in our datacenter.  It¹s a standard cluster
>> configuration:  IB network, distributed file system (BeeGFS.  I really
>> like it), Torque/Maui batch.  Our users run a varied workload, from
>> fine-grained, MPI-based parallel aps scaling to 100s of cores to
>> coarse-grained, high-throughput jobs (We¹re a CMS Tier-3 site) with high
>> I/O requirements.
>>
>> Whatever we transition to, whether it be a new in-house cluster or
>> something ³out there², I want to minimize the amount of change or learning
>> curve our users would have to experience.  They should be able to focus on
>> their research and not have to spend a lot of their time learning a new
>> system or trying to spin one up each time they have a job to run.
>>
>> If you have worked with HPC in the cloud, either as an admin and/or
>> someone who has used cloud resources for research computing purposes, I
>> would appreciate learning your experience.
>>
>> Even if you haven¹t used the cloud for HPC computing, please feel free to
>> share your thoughts or concerns on the matter.
>>
>> Sort of along those same lines, what are your thoughts about leasing a
>> cluster and running it on-site?
>>
>> Thanks for your time,
>>
>> Mike Hutcheson
>> Assistant Director of Academic and Research Computing Services
>> Baylor University
>>
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>> To change your subscription (digest mode or unsubscribe) visit
>> http://www.beowulf.org/mailman/listinfo/beowulf
>>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20150508/2eb45719/attachment.html>