[Beowulf] cloud: ho hum?

[Greg Keller]

<BIAS> I Sell HPC Cycles over the Internet </BIAS>

> Date: Wed, 1 Feb 2012 10:08:24 -0500 (EST)
> From: Mark Hahn <hahn at mcmaster.ca>
> Subject: [Beowulf] cloud: ho hum?
>        - private sector is inherently more efficient.  this is a bit
>        of a mystery to me, but I guess this is one of the great rhetorical
>        successes of the neocon movement.

It depends on who's accounting for what.  Businesses typically have to
include Power, Tax and Higher markups from Vendors for HW and SW than
Gov't and Academics.  Also NSF and others, at least on paper, require
them to charge no more than "cost" when selling cycles as I
understand.  So they should always be cheaper if the system is 100%
utilized over the life of the system.

>        I've looked at Amazon prices,
>        and they are remarkably high - depending on purchasing model,
>        about 20x higher than an academic-run research cluster.  why is there
>        not more skepticism of outsourcing, since it always means your cost
>        includes one or more corporate profit margins?

Please don't consider Amazon pricing "HPC in the Cloud"'s baseline or
norm.  Especially price/performance.  They do a great job at what they
do well, but in this instance they actually poison the market because
the price/performance is so bad for many workloads.

>
>        - economies of scale: people seem to think that a datacenter at the
>        scale of google/amazon/facebook is going to be dramatically cheaper.
>        while I'm sure they get a good deal from their suppliers, I also
>        doubt it's game-changing.  power, for instance, is a relatively
>        modest portion of costs, ~10% per year of a server's purchase price.
>        machineroom cost is pretty linear with number of nodes (power);
>        people overhead is very small (say, > 1000 servers per fte.)

There is also a significant penalty for any provider that builds
first, sells second that negates much of the "Economy of scale".  It's
like buying hard drive space 2 years in advance, if you wait and buy
in smaller chunks as needed you will end up with a lot more space over
the 2 years for the same spend.

>
> most of all, I just don't see how cloud changes the HPC picture at all.
> HPC is already based on shared resources handling burstiness of demand -
> if anything, cloud is simply slower.  certainly I can't submit a job to
> EC2 that uses half the Virgina zone and expect it to run immediately.
> it's not clear to me whether cloud-pushers are getting real traction with
> the funding agencies (gov is neocon here in Canada.)  it worries me that
> cloud might be framed as "better computing than HPC".
When done well, it's a continuation of a longer term trend:
researchers build their own cluster and don't share... then they put
them together in departments and team share... then those get pulled
into enterprise scale systems... then enterprises "share"
discretely/blindly through time-sharing at some external provider.
>
> I'm curious: what kind of cloudiness are you seeing?

Most organizations that have a large enough continuous baseline load
can probably save money doing the baseline in-house and "bursting" to
providers for special projects and anything that's speculative enough
that they may not be doing it for 3-5 years.  If your admin costs as
much as your cluster (because they read this list and are awesome) of
16 nodes you may be better off outsourcing even the baseline.
Ridiculous overhead, space, delay, or power costs can also help make
outsourcing HPC a better use for budget

If you outsource everything you don't have redundant providers you may
be adding risk to your organization to save a little money.  Once a
few providers that are independent support similar access and control
systems you could have redundant providers and shift workloads.  If
your competitor buys your current provider and shuts it down Oracle
style you still have a system to run on while you setup at a new
redundant provider.  This is IMHO the key limiter of HPC Outsourcing
growth and for good reasons.  Some of our early adopters have no
choice but to go external because the budget doesn't allow for a
purchase big enough to meet a short term project's needs, so the
redundancy risk of outsourcing is negated.

>
> thanks, mark hahn.
>

Also, a few points in reply to Chris...

> Date: Wed, 01 Feb 2012 10:37:30 -0500
> From: Chris Dagdigian <dag at sonsorol.org>
> Subject: Re: [Beowulf] cloud: ho hum?

> My $.02 from what I see in industry (life sciences)
>
> - The ability to transform capital expense money into OpEx money alone
> is pushing some cloud interest at high levels. No joke. Possibly a very
> large cloud interest driver in the larger organizations. This is also
> attractive for tiny startups and companies just leaving the VC
> incubation phase.
Very proven in our business experience.  Not just VC, even fortune 10
companies have projects that look like internal startups that may fail
in months.

>
> - Deployment speed. We have customers who wait weeks after making an IT
> helpdesk request for a new VM to be created. Other customers take 1+
> years to design, RFP and choose their HPC solution and another 4 months
> to deploy it.  If you can do in minutes (via good DevOps techniques)
> what the IT organization normally takes weeks or months to do then
> you've got some good arguments for targeting cloud environments for
> quick, dev, test and on-off scientific computing environments

We call this "Corporate Inertia".  Risk aversion by internal staff
makes obvious decisions committee decisions so no one person gets
blamed if there are complaints.

>
> - Quick capability gains - in some cases it's quicker and easier to get
> quick access to GPUs, servers with 10Gbe interconnects and well-built
> systems for running MapReduce style big data workflows on cloud platforms
>
> I agree that the cloud is overhyped and we certainly don't see a ton of
> HPC migrating entirely to the cloud. What we see in the trenches and out
> in the real world is significant interest in leveraging the cloud for
> Speed, Capability, Cost or "weird" use cases.

Agreed.  "Enterprise" cloud and "HPC" cloud aim at opposite purposes.
One is to subdivide a system for higher utilization (value), one is to
combine multiple systems for performance.  HPC benefits greatly from
transparent understanding of the actual hardware and configurations,
Enterprise benefits from not caring or needing to.  So "good" HPC
Clouds are Transparent whereas Enterprise clouds should only be
translucent.  You can't win a Nascar Race with $xM worth of
Convertible Geo Metro's, you Need a single $xM Nascar and a good pit
crew (admins).

>
>
> -Chrius
>

Cheers!
Greg
<BIAS> I Sell HPC Cycles over the Internet </BIAS>