[Beowulf] clustering using xen virtualized machines

[Tim Cutts]

On 28 Jan 2010, at 4:23 pm, Gavin Burris wrote:

> Sorry, I'm not drinking the virtualization/cloud koolaid.  I'd love to
> have everything abstracted and easy to manage, but I find  
> standardizing
> on an OS or two and keeping things as stock as possible is easier, and
> cheaper to manage at this point.  In my situation, virtualization just
> adds complexity and has a price/performance penalty.

For HPC, I think you're probably right at the moment.  But for more  
run-of-the-mill servers, the cost/benefit and simplification is  
definitely good.  I've got about 9 physical servers providing virtual  
machines for web servers, development nodes, infrastructure services  
(mail, and so on) and all that stuff.  Those 9 servers are running 170  
virtual machines, which in the old days would mostly have been  
separate boxes, and in many cases redundant pairs for failover.  OK,  
so the servers are meatier, and cost maybe four times what the basic  
tin we'd have used would have cost for a single server.  I still make  
that about 75% less money on hardware than we would otherwise have  
spent for that number of services.  The saving is much larger than  
what it cost to buy vSphere.  Power consumption is more like a 90%  
saving - my entire virtualisation setup consumes about 3.6kW (not  
counting the storage), which is, what, about 20W per VM (and we're not  
full yet).  And it all sits in 9U of rack space.

And then there are all the fringe benefits which save me time;  
simplified storage allocation, reduced deployment time, almost  
complete elimination of service downtime for hardware maintenance,  
guest OS patch management and automated remediation (for Windows, SLES  
and RHAS anyway - most of our machines run Debian which sadly they  
don't do patch management for).  I get HA for free, so I no longer  
have to fart about with heartbeat and redundant server pairs.   I get  
lock-step fault tolerance for free, too, if I need it, so I can  
finally get rid of that Marathon abomination.  Backups become simpler  
(meh, just back up the whole VM with Consolidated Backup).

You're still right that the management of VM setup takes quite a lot  
of time, but it's a lot less than if I were having to configure and  
deploy the same wide variety of services on physical hardware.

But Cloud stuff, I'm right with you and slightly skeptical at the  
moment.  Especially for our extremely data-heavy CPU-lite  
applications.  It's more likely to have application in our line of  
work for the ability to ship arbitrary untrusted code to data.  My  
dream world is for all the sequencing sites to present their data to a  
cloud interface in a consistent manner, and if I want to analyse, say,  
Broad's data, I just ship my VM to them and run my analysis there.   
Similarly, we provide hosts for running their VMs.  No more shipping  
disks around by Fedex, which is what the scientists currently do.  But  
it's probably never going to happen.  *sigh*.  Far too much "not  
invented here" politics.

Regards,

Tim




-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.