[Beowulf] Virtualized Infiniband in KVM?

[Christopher Samuel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 29/03/11 01:18, Joe Landman wrote:

> I know about the VT-d bits for virtualized 1 1GbE/10GbE
> and their  ability to be leveraged for KVM stacks.  Is
> there a similar capability for IB?

I believe so, but according to this presentation the
numbers don't look that good:

http://salsahpc.indiana.edu/CloudCom2010/slides/PDF/Recommendations%20for%20Virtualization%20Technologies%20in%20High%20Performance%20Computing.pdf

It was presented on 2nd December 2010 so probably
reasonably up to date although their slides provide
no information at all on which kernel version they
were using for this testing.

More info on the paper here:

http://salsahpc.indiana.edu/CloudCom2010/papers.html#00108

[...]
# This work benchmarks two virtual machine monitors, OpenVZ and KVM,
# specifically focusing on I/O throughput since CPU efficiency has
# been extensively studied [1]. OpenVZ offers near native I/O
# performance. Amazon’s EC2 “Cluster Compute Node” product is also
# considered for comparative purposes and performs quite well. The
# EC2 “Cluster Compute Node” product utilizes the Xen hypervisor in
# hvm mode and 10 Gbit/s Ethernet for high throughput communication.
# Therefore, we also briefly studied Xen on our hardware platform
# (in hvm mode) to determine if there are still areas of improvement
# in KVM that allow EC2 to outperform KVM (with InfiniBand host channel
# adapters operating at 20 Gbit/s) in MPI benchmarks. We conclude
# that KVM’s I/O performance is suboptimal, potentially due to memory
# management problems in the hypervisor. Amazon’s EC2 service is
# promising, although further investigation is necessary to understand
# the effects of network based storage on I/O throughput in compute
# nodes. Amazon’s offering may be attractive for users searching for
# "InfiniBand-like" performance without the upfront investment required
# to build an InfiniBand cluster or users wishing to dynamically expand
# their cluster during periods of high demand.

cheers!
Chris
- -- 
    Christopher Samuel - Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
         http://www.vlsci.unimelb.edu.au/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2UXdkACgkQO2KABBYQAh+O+wCeKazajm8BAoVPKFtS3LrxF3BE
SxEAniyVrwxyQTzm+oLy8ny+aPqTilxh
=qByj
-----END PGP SIGNATURE-----