[Beowulf] Virtualization in head node ?

[Tim Cutts]

On 15 Sep 2009, at 11:55 pm, Dmitry Zaletnev wrote:

> When install CentOS 5.3, you get Xen virtual machine for free, with  
> a nice interface, and in it, modes with internal network and NAT to  
> outside world work simultaneously, witch is not the case of Sun xVM  
> VirtualBox. Never used VMWare because of its value of $189, people  
> say it's a good VM. But whatfor, if there is CentOS 5.3 with Xen,  
> the industry best emulator/VM?

VMware has some free versions; the pay-for versions have a number of  
extra features which are generally missing from the competitors, and  
some of which are quite shiny.  The automated hot migration of VMs to  
load balance, for example.  Last time I looked, you could manually  
migrate Xen VMs from one host to another, but it wouldn't do it  
automatically.  vSphere also has high availability and fault tolerance  
features; I use the HA but not the FT yet (FT is like Marathon for  
Windows - it runs two copies of the VM in lock-step on two hosts, so  
that if one of the physical servers dies, the VM doesn't even need to  
reboot.  Obviously there's a significant performance penalty in this).

The other thing I find useful in vSphere that isn't yet present in Xen  
(at least last time I looked) was the ability to give particular users  
fine-grained access to their VM.  For example, I used to have to give  
some users sudo access to their machines, and generally I can get  
around that now by allowing them to reboot their virtual machine  
instead, and they no longer need sudo at all.  I consider this an  
improvement.

More contentious is the memory deduplication trick.  I can see  
arguments both for and against this.  VMware's workstations products,  
and Xen, and presumably other hypervisors, give the VM as much RAM as  
you configure it with, regardless of whether it's going to use it or  
not.  ESX can be configured to do this too, but by default it doesn't,  
and allows you to overcommit memory.  It pays for this partly by  
deduplicating memory pages.  Here's the output from the esxtop monitor  
program on one of our VMware servers (an HP BL490 blade server with  
72GB of RAM):

  3:29:06pm up 6 days 23:40, 161 worlds; MEM overcommit avg: 0.00,  
0.00, 0.00
PMEM  /MB: 73718   total:   618     cos,   967 vmk,   30880 other,   
41252 free
VMKMEM/MB: 72164 managed:  4329 minfree,  5980 rsvd,  65700 ursvd,   
high state
COSMEM/MB:    69    free:  1239  swap_t,  1239 swap_f:   0.00 r/s,    
0.00 w/s
NUMA  /MB: 36582 (11703), 36034 (29548)
PSHARE/MB:  3953  shared,   121  common:  3832 saving
SWAP  /MB:    13    curr,     0  target:                 0.00 r/s,    
0.00 w/s
MEMCTL/MB:     0    curr,     0  target, 25748 max

The PSHARE row is the key one here; it's identified 3953 MB of memory  
pages which are the same on various machines, and is using only 121 MB  
to store them, saving 3832 MB of RAM.  Our VMs are very heterogeneous;  
there are CentOS, Scientific Linux, Debian 4, Debian 5, SLES 10 SP2,  
Windows XP (both 32 and 64-bit), Windows Server 2003 (both 32 and 64- 
bit), Solaris... if they were more homogeneous, I'm sure the PSHARE  
saving would be much higher.

Tim


-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.