[Beowulf] Clustering VPS servers

[Joe Landman]

On 03/21/2013 12:15 PM, Jonathan Aquilina wrote:
> I am venturing back to the cloud side again isn't cloud just a fancy name
> for virtualized servers?

Well, no.

Think of cloud as "hardware and infrastructure as code".  You 
instantiate what you "need" with some caveats.

Completeness of API, breadth of product is nice, but at the end of the 
day, the only thing that really matters, is, does it do what you need at 
a price you are willing to pay.

+1 to Chris for message going over these elements, though one has to be 
careful as Amazon isn't the only one out there.  They are good for some 
things, but as many of our customers have discovered by trying them, not 
good for everything, nor even the majority of items.  Clouds (public and 
private) have a definite (and growing) niche, and I see the entire 
"hardware and infrastructure as code" as a very good thing.  It has a 
nasty tendency to create waves of fads though, and its important to be 
able to recognize such things.

This said, not having a block, object, or other storage API when you 
don't need it really isn't a deal killer.  Not having an end-to-end low 
latency fabric, and bare metal servers is (for our customers). But our 
customers are not necessarily the same as all customers, and the 
recursive joke I tell about this is "gross overarching generalizations 
tend to be incorrect".  Basically being all things to all people means 
that none are done well.  Focusing upon niches where people get what 
they need is IMO a good way to carve out a market.

With all due respect to those whom advocate heavily for one thing or the 
other, our experience with this is that the joke is on the end user when 
things don't work well at all, and cannot be made to work well due to 
design/implementation issues.  Especially if they've sold this to 
management/board of directors.  This is true of internal as well as 
external resources.  This is more of a generic business problem than a 
public cloud vs private resource issue.

This said, a quick pointer over to an IDC paper is an interesting read: 
http://www.idc.com/getdoc.jsp?containerId=prUS23972413#.UUs7OqDYSlg is a 
good read.  Also there are others as well.  Public cloud is large and 
growing, and largely commiditized.  Very non-HPC specific, though with 
careful marshalling/selection of resources (CycleComputing) it can be 
viable for some HPC.  But its very hard to argue a single platform is 
all things to all people, when it clearly isn't.  Moreover, Amazon 
itself just signed a nice sized deal with a government entity for a 
private cloud offering ...

Basically, Chris is dead on right with his points on "there must be an 
API, and a broad array of services" for a good public cloud.  A private 
cloud can get more specific on needs, and not worry about implementing 
things that are not needed right away.

But cloud isn't always "virtualized".  Amazon is largely 
para-virtualized which is great to maximize tenancy, but not so great 
for performance.  Its APIed out the wahzoo.  Other clouds are 
hyper-virtualized and run closer to the metal.  Others still are 
bare-metal with provisioning magic atop them.

What likely matters most with public clouds are the ability to move 
between them when one of your cloud providers decides to compete with 
you.  This has been a thread of articles on The Register (which I wrote 
about here: http://scalability.org/?p=5898 )

This isn't an Amazon issue as much as it is a business dependency 
issue.  Freedom to move is important.  It would be nice to have a common 
API so as to make this work well.  The very last thing any of us want is 
the Microsoft-ization of the cloud world.  That would be, universally, 
and catastrophically, bad for all.
<http://scalability.org/?p=5898>

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/siflash
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615