[Beowulf] Clustering VPS servers

[Jonathan Aquilina]

Thing with linode is that they focus on infrastructure as a service in the
sense they have 6 dc's globally and can easily deply redeploy migrate clone
backup load balance with ease and very reasonably priced.

-----Original Message-----
From: Joe Landman [mailto:landman at scalableinformatics.com] 
Sent: 21 March 2013 18:11
To: Jonathan Aquilina
Cc: beowulf at beowulf.org
Subject: Re: [Beowulf] Clustering VPS servers

On 03/21/2013 12:15 PM, Jonathan Aquilina wrote:
> I am venturing back to the cloud side again isn't cloud just a fancy 
> name for virtualized servers?

Well, no.

Think of cloud as "hardware and infrastructure as code".  You instantiate
what you "need" with some caveats.

Completeness of API, breadth of product is nice, but at the end of the day,
the only thing that really matters, is, does it do what you need at a price
you are willing to pay.

+1 to Chris for message going over these elements, though one has to be
careful as Amazon isn't the only one out there.  They are good for some
things, but as many of our customers have discovered by trying them, not
good for everything, nor even the majority of items.  Clouds (public and
private) have a definite (and growing) niche, and I see the entire "hardware
and infrastructure as code" as a very good thing.  It has a nasty tendency
to create waves of fads though, and its important to be able to recognize
such things.

This said, not having a block, object, or other storage API when you don't
need it really isn't a deal killer.  Not having an end-to-end low latency
fabric, and bare metal servers is (for our customers). But our customers are
not necessarily the same as all customers, and the recursive joke I tell
about this is "gross overarching generalizations tend to be incorrect".
Basically being all things to all people means that none are done well.
Focusing upon niches where people get what they need is IMO a good way to
carve out a market.

With all due respect to those whom advocate heavily for one thing or the
other, our experience with this is that the joke is on the end user when
things don't work well at all, and cannot be made to work well due to
design/implementation issues.  Especially if they've sold this to
management/board of directors.  This is true of internal as well as external
resources.  This is more of a generic business problem than a public cloud
vs private resource issue.

This said, a quick pointer over to an IDC paper is an interesting read: 
http://www.idc.com/getdoc.jsp?containerId=prUS23972413#.UUs7OqDYSlg is a
good read.  Also there are others as well.  Public cloud is large and
growing, and largely commiditized.  Very non-HPC specific, though with
careful marshalling/selection of resources (CycleComputing) it can be viable
for some HPC.  But its very hard to argue a single platform is all things to
all people, when it clearly isn't.  Moreover, Amazon itself just signed a
nice sized deal with a government entity for a private cloud offering ...

Basically, Chris is dead on right with his points on "there must be an API,
and a broad array of services" for a good public cloud.  A private cloud can
get more specific on needs, and not worry about implementing things that are
not needed right away.

But cloud isn't always "virtualized".  Amazon is largely para-virtualized
which is great to maximize tenancy, but not so great for performance.  Its
APIed out the wahzoo.  Other clouds are hyper-virtualized and run closer to
the metal.  Others still are bare-metal with provisioning magic atop them.

What likely matters most with public clouds are the ability to move between
them when one of your cloud providers decides to compete with you.  This has
been a thread of articles on The Register (which I wrote about here:
http://scalability.org/?p=5898 )

This isn't an Amazon issue as much as it is a business dependency issue.
Freedom to move is important.  It would be nice to have a common API so as
to make this work well.  The very last thing any of us want is the
Microsoft-ization of the cloud world.  That would be, universally, and
catastrophically, bad for all.
<http://scalability.org/?p=5898>

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/siflash
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615