[Beowulf] Home beowulf - NIC latencies
patrick at myri.com
Sat Feb 5 18:27:57 PST 2005
Vincent Diepeveen wrote:
>>>CPU's are 100% busy and after i know how many times a second the network
>>>can handle in theory requests i will do more probes per second to the
>>>hashtable. The more probes i can do the better for the game tree search.
>>With a gigE network that sounds like 40us or so. With Myrinet or IB
>>it's in the 4-6us range. If you bought dual opterons with the special
> At the quadrics and dolphin homepage they both claim 12+ us for Myrinet.
Seriously, here are MPI latencies with MX on F cards on Opteron (PCI-X),
that includes fibers and a switch in the middle:
Length Latency(us) Bandwidth(MB/s)
0 2.684 0.000
1 2.874 0.336
2 2.898 0.690
4 2.978 1.343
8 2.965 2.699
16 2.993 5.347
32 3.409 9.388
64 3.563 17.960
128 3.977 32.185
256 5.699 44.916
Quadrics would be lower by a 1.5 us, I don't know about Dolphin, I
didn't hear about noticeable SCI clusters in a long time.
> I am very impressed by the quadrics and dolphin cards. Probably by
> infinipath too when i check them out. Will do.
> I'm not so impressed yet by myrinet actually, but if cluster builders can
> earn a couple of hundreds of dollars more on each node i'm sure they'll do it.
I don't think Myrinet would be the cheapest, I am sure you can get a
better deal from desperate interconnect vendors.
What does not impress you in Myrinet ?
More information about the Beowulf