[Beowulf] Clustering VPS servers

[Chris Dagdigian]

Jonathan Aquilina wrote:
>
> It’s not that I need to cluster these vps’s I was just wondering if it 
> was possible. What puts me off about amazon is pricing. It seems a bit 
> pricy so to speak.
>

MIT StarCluster (the open source stack that builds Grid Engine clusters 
on Amazon mentioned elsewhere in this thread) is able to leverage the 
AWS Spot Market and the potential savings off of the hourly EC2 rate is 
pretty enormous. Via Spot you can run servers for pennies an hour that 
traditionally sell for dollars-per-hour on the "normal" EC2 on-demand 
service. It would be very hard to beat that price on an internal 
infrastructure if one was honest about the fully loaded facility, energy 
and staffing costs.

I ran 30-40 of Amazon's biggest and newest compute cluster instance 
types + 16 of their GPU nodes on the Spot Market and a few persistent 
license and control servers to manage the workload and I think our 
hourly cost was less than $30/hour. We only paid that rate for the 
duration of the simulation then tore everything down. Totally worth it 
and we ran at a scale that would not have been possible internally.

I was originally leery of the Spot Market thinking that it would be too 
disruptive to have nodes potentially yanked from under me but it turns 
out the pricing history is easily accessible and you can see that over 
time the prices are usually super stable. I've been able to pay pennies 
for decent sized servers and run them for weeks at a time.

However there are occasionally spikes and it's interesting to see how 
the arbitrage/bidding strategies work. I've been doing some 
low-importance analysis on a c1.medium server recently and I ran for a 
few days at just a few pennies an hour before my server was blown away 
due to the spot price rising higher than my max bid ($0.11) . Looking at 
the pricing history there was a brief 1-hour window of time when the 
spot price for a c1.medium instance type in region us-east-1b spiked to 
*several dollars* -- far higher than even the hourly on-demand rate that 
Amazon charges for the same server.

One piece of advice is to avoid us-east-1b -- that is my personal 
default and I think many others do the same. Spot price variability 
seems less volatile in the other zones.

My $.02!

-Chris