[Beowulf] Virtualisation and high performance interconnects.

[Robert G. Brown]

On Thu, 1 Nov 2007, Mark Hahn wrote:

>> However, it does for all non-OS-bypass operations. The most important of 
>> these is memory registration, because it's not a one-time cost as the Ohio 
>> State papers have always portrayed it. For applications that do not
>
> it's also worth pointing out that virtualization introduces some amount of 
> jitter to a parallel application's timing.  this will not
> matter at all if the app is fairly small and/or loosely-coupled.
> but for nontrivial sized jobs that message frequently or in large
> collectives...

This is absolutely true, and is easily visible -- I rather mean audible
-- if you try to play music under a VM.  It "works" in the sense that
WMP under XP-Pro will dutifully play mp3s or a cast of some sort, but
the jitter is clearly audible as a mix of "noise" from very short holes
inserted into the play interspersed with gaps when you lose enough
cycles to actually break into the decoding process.

I'm very enthusiastic about VMs for use both in desktop or server
environments (where the advantages are profound and can be very
cost-beneficial) and in certain clustering environments or problems, but
even if you view the application space for the computer as being "flat"
in some sense, there are just more processes competing for cycles when
you have two fat OS layers running instead of on thin one alongside your
application.  A lot of bookkeeping gets done twice, two schedulers are
running (one slicing a slice of the other), they both have various
running services independently, and there is also VMware or whatever
eating cycles of its own managing the interface.

The NUMBER of burned cycles may not be all that significant, if the host
and guest are both mostly idle except for the primary running process,
but the timing becomes far noisier.

I don't know how many of these problems are really "fundamental" though.
I wish I knew more about the hardware support for VMs.  All I know is
that current chips have it, but I don't know what it consists of.  Does
anybody?  Want to share?

     rgb

> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf
>

-- 
Robert G. Brown
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone(cell): 1-919-280-8443
Web: http://www.phy.duke.edu/~rgb
Lulu Bookstore: http://stores.lulu.com/store.php?fAcctID=877977