Intial Pallas performance with Myrinet on a 860 & E7500
alex at compusys.co.uk
alex at compusys.co.uk
Thu Apr 18 01:58:16 PDT 2002
For your information, please look at the following performance
measurements for the 'C' class Myrinet2000 cards.
Details of the two machines (optimisation level: -fast & PGI):
- 2.4.17 kernel
- mpich-1.2.1..7b
- gm-1.5.1
- measurement performed between machines
860 Supermicro DCE:
- Dual P4 2 GHz
- C class Myrinet2000
The new E75000 Supermicro DDR :
- Dual P4 1.8GHz
- C class Myrinet2000, using PCI-X slot
Notice the results for E75000 Sendrecv (and Exchange):
4194304 --> 290.29Mbytes/s
That is more than the serverworks LE chipset.
Alex
(shown results are limited due to mailing list size limit)
///////////////////////// E75000 //////////////////////////////////////
#---------------------------------------------------
# Benchmarking PingPong
# ( #processes = 2 )
#---------------------------------------------------
#bytes #repetitions t[usec] Mbytes/sec
0 1000 8.79 0.00
1 1000 8.96 0.11
2 1000 8.94 0.21
4 1000 8.96 0.43
8 1000 8.96 0.85
16 1000 9.03 1.69
32 1000 9.32 3.27
64 1000 9.44 6.47
128 1000 12.14 10.06
2097152 20 8726.80 229.18
4194304 10 17300.95 231.20
#---------------------------------------------------
# Benchmarking PingPing
# ( #processes = 2 )
#---------------------------------------------------
#bytes #repetitions t[usec] Mbytes/sec
0 1000 11.97 0.00
1 1000 12.64 0.08
2 1000 12.09 0.16
4 1000 12.87 0.30
8 1000 12.33 0.62
16 1000 12.36 1.23
32 1000 11.73 2.60
64 1000 11.90 5.13
128 1000 14.65 8.33
2097152 20 13792.25 145.01
4194304 10 27535.80 145.27
#-----------------------------------------------------------------------------
# Benchmarking Sendrecv
# ( #processes = 2 )
#-----------------------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec] Mbytes/sec
0 1000 12.60 12.60 12.60 0.00
1 1000 12.68 12.69 12.68 0.15
2 1000 12.76 12.76 12.76 0.30
4 1000 12.73 12.73 12.73 0.60
8 1000 12.52 12.53 12.53 1.22
16 1000 12.59 12.59 12.59 2.42
32 1000 11.74 11.74 11.74 5.20
64 1000 11.81 11.81 11.81 10.34
128 1000 14.41 14.42 14.42 16.93
2097152 20 13778.64 13778.80 13778.72 290.30
4194304 10 27558.40 27558.70 27558.55 290.29
#-----------------------------------------------------------------------------
# Benchmarking Exchange
# ( #processes = 2 )
#-----------------------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec] Mbytes/sec
0 1000 20.93 20.93 20.93 0.00
1 1000 20.96 20.97 20.97 0.18
2 1000 20.96 20.97 20.96 0.36
4 1000 20.96 20.97 20.97 0.73
8 1000 21.03 21.05 21.04 1.45
16 1000 21.01 21.02 21.02 2.90
32 1000 21.30 21.30 21.30 5.73
64 1000 21.33 21.34 21.33 11.44
128 1000 23.98 23.98 23.98 20.36
2097152 20 27563.40 27563.60 27563.50 290.24
4194304 10 55052.30 55053.10 55052.70 290.63
#----------------------------------------------------------------
# Benchmarking Allreduce
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 0.15 0.15 0.15
4 1000 19.22 19.23 19.23
8 1000 19.25 19.26 19.25
16 1000 19.31 19.33 19.32
32 1000 20.02 20.03 20.03
64 1000 20.39 20.40 20.40
128 1000 25.96 25.97 25.96
2097152 20 30421.15 30422.15 30421.65
4194304 10 67887.70 67889.70 67888.70
#----------------------------------------------------------------
# Benchmarking Reduce
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 0.08 0.08 0.08
4 1000 10.01 10.02 10.01
8 1000 10.01 10.02 10.01
16 1000 10.06 10.07 10.07
32 1000 10.40 10.41 10.40
64 1000 10.62 10.63 10.63
128 1000 14.25 14.26 14.25
2097152 20 25153.70 25231.65 25192.67
4194304 10 49852.60 50856.40 50354.50
#----------------------------------------------------------------
# Benchmarking Reduce_scatter
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 0.53 0.55 0.54
4 1000 21.00 21.00 21.00
8 1000 21.39 21.40 21.39
16 1000 21.29 21.30 21.29
32 1000 21.70 21.71 21.70
64 1000 22.37 22.38 22.37
128 1000 25.39 25.40 25.40
2097152 20 41247.20 41436.80 41342.00
4194304 10 70550.10 70943.20 70746.65
#----------------------------------------------------------------
# Benchmarking Allgather
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 12.16 12.16 12.16
1 1000 12.61 12.61 12.61
2 1000 12.52 12.52 12.52
4 1000 12.41 12.41 12.41
8 1000 12.69 12.69 12.69
16 1000 12.71 12.71 12.71
32 1000 13.26 13.27 13.26
64 1000 13.06 13.06 13.06
128 1000 17.72 17.73 17.72
2097152 20 23349.75 23350.30 23350.03
4194304 10 38150.00 38151.60 38150.80
#----------------------------------------------------------------
# Benchmarking Allgatherv
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 12.48 12.48 12.48
1 1000 12.41 12.42 12.42
2 1000 12.16 12.17 12.16
4 1000 12.21 12.21 12.21
8 1000 12.52 12.52 12.52
16 1000 12.35 12.35 12.35
32 1000 13.09 13.09 13.09
64 1000 12.80 12.80 12.80
128 1000 17.19 17.19 17.19
2097152 20 19057.95 19058.55 19058.25
4194304 10 37964.00 37965.39 37964.70
#----------------------------------------------------------------
# Benchmarking Alltoall
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 12.44 12.44 12.44
1 1000 12.72 12.72 12.72
2 1000 12.50 12.50 12.50
4 1000 12.56 12.56 12.56
8 1000 12.56 12.56 12.56
16 1000 12.75 12.75 12.75
32 1000 12.86 12.86 12.86
64 1000 13.73 13.73 13.73
128 1000 17.87 17.87 17.87
2097152 20 19927.10 19927.60 19927.35
4194304 10 39608.10 39609.79 39608.94
#----------------------------------------------------------------
# Benchmarking Bcast
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 0.09 0.09 0.09
1 1000 9.09 9.09 9.09
2 1000 9.07 9.08 9.07
4 1000 9.06 9.07 9.07
8 1000 9.09 9.10 9.09
16 1000 9.15 9.16 9.16
32 1000 9.44 9.44 9.44
64 1000 9.56 9.57 9.56
128 1000 11.63 11.64 11.63
2097152 20 8740.05 8740.20 8740.12
4194304 10 17313.69 17313.90 17313.80
#---------------------------------------------------
# Benchmarking Barrier
# ( #processes = 2 )
#---------------------------------------------------
#repetitions t_min[usec] t_max[usec] t_avg[usec]
1000 12.45 12.45 12.45
/////////////////////////// 860 //////////////////////////////////
#---------------------------------------------------
# Benchmarking PingPong
# ( #processes = 2 )
#---------------------------------------------------
#bytes #repetitions t[usec] Mbytes/sec
0 1000 8.93 0.00
1 1000 9.14 0.10
2 1000 9.17 0.21
4 1000 9.14 0.42
8 1000 9.41 0.81
16 1000 9.54 1.60
32 1000 9.85 3.10
64 1000 10.06 6.06
128 1000 12.77 9.56
2097152 20 11924.45 167.72
4194304 10 23752.65 168.40
#---------------------------------------------------
# Benchmarking PingPing
# ( #processes = 2 )
#---------------------------------------------------
#bytes #repetitions t[usec] Mbytes/sec
0 1000 11.94 0.00
1 1000 12.47 0.08
2 1000 12.83 0.15
4 1000 13.02 0.29
8 1000 12.41 0.61
16 1000 12.82 1.19
32 1000 12.07 2.53
64 1000 12.27 4.98
128 1000 14.50 8.42
2097152 20 21075.20 94.90
4194304 10 42104.29 95.00
#-----------------------------------------------------------------------------
# Benchmarking Sendrecv
# ( #processes = 2 )
#-----------------------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec] Mbytes/sec
0 1000 12.20 12.21 12.20 0.00
1 1000 12.83 12.84 12.83 0.15
2 1000 12.85 12.85 12.85 0.30
4 1000 12.61 12.62 12.62 0.60
8 1000 12.63 12.63 12.63 1.21
16 1000 12.55 12.55 12.55 2.43
32 1000 12.40 12.40 12.40 4.92
64 1000 12.70 12.70 12.70 9.61
128 1000 14.54 14.55 14.55 16.78
2097152 20 21075.50 21075.85 21075.67 189.79
4194304 10 42110.90 42112.10 42111.50 189.97
#-----------------------------------------------------------------------------
# Benchmarking Exchange
# ( #processes = 2 )
#-----------------------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec] Mbytes/sec
0 1000 21.23 21.23 21.23 0.00
1 1000 21.56 21.63 21.59 0.18
2 1000 21.56 21.57 21.56 0.35
4 1000 21.49 21.49 21.49 0.71
8 1000 21.63 21.63 21.63 1.41
16 1000 21.68 21.68 21.68 2.81
32 1000 21.87 21.88 21.88 5.58
64 1000 22.16 22.16 22.16 11.02
128 1000 24.66 24.66 24.66 19.80
2097152 20 42154.20 42155.05 42154.62 189.78
4194304 10 84224.61 84225.20 84224.90 189.97
#----------------------------------------------------------------
# Benchmarking Allreduce
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 0.15 0.16 0.16
4 1000 19.93 19.94 19.94
8 1000 20.31 20.33 20.32
16 1000 20.77 20.78 20.78
32 1000 21.54 21.55 21.55
64 1000 21.96 21.97 21.96
128 1000 26.05 26.06 26.05
2097152 20 36295.15 36300.15 36297.65
4194304 10 72057.59 72060.60 72059.09
#----------------------------------------------------------------
# Benchmarking Reduce
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 0.09 0.09 0.09
4 1000 10.48 10.49 10.49
8 1000 10.92 10.93 10.92
16 1000 11.03 11.04 11.03
32 1000 11.40 11.41 11.40
64 1000 11.65 11.66 11.65
128 1000 13.85 13.86 13.86
2097152 20 24145.65 24442.65 24294.15
4194304 10 47357.39 48542.51 47949.95
#----------------------------------------------------------------
# Benchmarking Reduce_scatter
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 0.65 0.66 0.66
4 1000 22.04 22.05 22.05
8 1000 22.72 22.73 22.73
16 1000 23.31 23.32 23.31
32 1000 23.89 23.90 23.90
64 1000 24.45 24.46 24.45
128 1000 26.94 26.95 26.94
2097152 20 33828.60 33844.10 33836.35
4194304 10 67314.90 67377.90 67346.40
#----------------------------------------------------------------
# Benchmarking Allgather
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 12.16 12.16 12.16
1 1000 12.32 12.32 12.32
2 1000 12.33 12.33 12.33
4 1000 12.36 12.36 12.36
8 1000 12.61 12.61 12.61
16 1000 12.71 12.71 12.71
32 1000 13.04 13.04 13.04
64 1000 13.86 13.86 13.86
128 1000 17.59 17.59 17.59
2097152 20 26607.50 26608.05 26607.78
4194304 10 53338.10 53338.91 53338.50
#----------------------------------------------------------------
# Benchmarking Allgatherv
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 12.18 12.18 12.18
1 1000 12.35 12.35 12.35
2 1000 12.30 12.30 12.30
4 1000 12.32 12.32 12.32
8 1000 12.52 12.53 12.53
16 1000 12.91 12.91 12.91
32 1000 13.11 13.11 13.11
64 1000 13.73 13.73 13.73
128 1000 17.58 17.58 17.58
2097152 20 26836.70 26838.00 26837.35
4194304 10 53090.61 53091.80 53091.20
#----------------------------------------------------------------
# Benchmarking Alltoall
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 12.86 12.86 12.86
1 1000 12.85 12.85 12.85
2 1000 13.19 13.19 13.19
4 1000 13.01 13.01 13.01
8 1000 13.23 13.23 13.23
16 1000 13.43 13.44 13.44
32 1000 13.78 13.78 13.78
64 1000 14.41 14.41 14.41
128 1000 18.18 18.18 18.18
2097152 20 27169.85 27170.25 27170.05
4194304 10 54303.90 54304.40 54304.15
#----------------------------------------------------------------
# Benchmarking Bcast
# ( #processes = 2 )
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 0.07 0.10 0.08
1 1000 9.29 9.30 9.29
2 1000 9.23 9.24 9.24
4 1000 9.25 9.26 9.26
8 1000 9.54 9.55 9.55
16 1000 9.69 9.69 9.69
32 1000 9.96 9.98 9.97
64 1000 10.14 10.15 10.15
128 1000 12.21 12.22 12.21
2097152 20 11937.15 11937.50 11937.32
4194304 10 23764.00 23764.71 23764.35
#---------------------------------------------------
# Benchmarking Barrier
# ( #processes = 2 )
#---------------------------------------------------
#repetitions t_min[usec] t_max[usec] t_avg[usec]
1000 12.90 12.90 12.90
More information about the Beowulf
mailing list