[Beowulf] IB DDR: mvapich2 vs mvapich performance

[Tom Elken]

> [mailto:beowulf-bounces at beowulf.org] On Behalf Of Mikhail Kuzminsky
> I have up to 1453 MB/s for 4MB message size ... on osu_bw test
w/Mellanox DDR IB 
> (Mellanox version of OFED-1.2, w/binary mvapich-0.9.9); OpenMPI
-1.2.2-1 gives even a 
> bit more (1470 MB/s - more exactly, 1469.753328,  1469.447179, 
> 1469.977840 for 3 subsequent test runs).
> 
> The SC'07 message of D.K.Panda
> http://mvapich.cse.ohio-state.edu/publications/sc07_mpich2_bof.pdf
> inform us about 1405 MB/s.
> 
> Is this throughput difference the result of MPI-2 vs MPI 
> implementation or should I beleive that this difference (about 4% for 
> my mvapich vs mvapich2 at SC'07 ) is not significant  - in the sense 
> that it is simple because of some measurement errors (inaccuracies)? 

The way to see if there is a real throughput difference between a MPI-2
implementation and a MPI-1 implementation is to measure it on your pair
of machines.

Comparing your results to published results are difficult because nearly
all the variables need to be the same for the comparison to be valid.
Variables like which of the following were used in the two tests:
- Mellanox IB DDR adapter
- PCIe interface type
- CPU model and speed
- PCIe chipset
- OFED version, 
...

Certainly the MPI flavor and version is important, but it is not, in
general, the most important of these factors.
Note for example these two results on the OSU MVAPICH web pages:

MVAPICH2  1-sided put throughput, measured with osu_bw:

1405 MB/s:  ConnectX DDR, PCIe x8, EM64T 2.33 GHz quad-core CPU
http://mvapich.cse.ohio-state.edu/performance/mvapich2/em64t/MVAPICH2-em
64t-gen2-ConnectX-DDR-1S.shtml

1481 MB/s:  MT25208 HCA silicon, PCIe x8, Intel Xeon 3.6 Ghz, EM64T

http://mvapich.cse.ohio-state.edu/performance/mvapich2/em64t/MVAPICH2-em
64t-gen2-DDR-1S.shtml

Both are DDR IB adapters.  ConnectX is the newer silicon.  But because
of system differences, the older adapter is faster, in this case.

-Tom


> 
> Mikhail Kuzminsky
> Computer Assistance to Chemical Research Center
> Zelinsky Institute of Organic Chemistry
> Moscow
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) 
> visit http://www.beowulf.org/mailman/listinfo/beowulf
>