low-latency high-bandwidth OS bypass user-level messaging for commodity(linux) clusters with commodity NICs(<$200), HELP! (GAMMA/EMP/M-VIA/etc.)
jcmcknny at uiuc.edu
Mon Dec 16 22:12:30 PST 2002
> -----Original Message-----
> From: Donald Becker [mailto:becker at scyld.com]
> Sent: Monday, December 16, 2002 6:32 PM
> To: jon
> Cc: beowulf at beowulf.org
> Subject: Re: low-latency high-bandwidth OS bypass user-level messaging
> commodity(linux) clusters with commodity NICs(<$200), HELP!
> On Mon, 16 Dec 2002, jon wrote:
> > Perhaps this isn't the best way to get ahold of you, but I've also
> > this to the Beowulf list. I've noted your comments on OS Bypass
> > in the past. But isn't there some room for non-TCP/IP related
> > such as with computing clusters? We don't need no stinking TCP! No
> > associated revenue? You could replace Myrinet in the thousands of
> > we have here ALONE at NCSA.
> Very likely not. Myrinet has both a cost and increasing performance
> advantage over gigabit Ethernet when the switch is larger than about
[jon] I see.
> > We (UIUC theoretical astrophysics group) are in the midst of
> > a $50K cluster (I know, small, but big for us! :)) and I'm done all
> > research as to what we should be getting. We ended up going with a
> > Intel Desktop gigabit board and P4, but have found the tests to be
> > poor. We only have 4 nodes right now because we worried about this
> > thing.
> Latency or bandwidth? And what are you using to test?
[jon] We need (projected 0 message size) latencies to be about 30-40us
for our applications. We end up with message sizes ranging from
256bytes to 8KB for different applications. A 2.4Ghz P4 cluster has an
idling CPU due to the latency.
> While there are better Gigabit chips than the DP83820, most of its bad
> reputation comes from the poor performance of the other drivers out
> there. We get quite reasonable performance from it with the Scyld
> ns820.c driver. Others have reported a 2.5-3X performance improvement
> over the driver written by Red Hat.
[jon] I see, what kinda of 0message latency and peak bandwidth do you
get on 64-bit 66Mhz bus?
[jon] Thanks! -Jon
More information about the Beowulf