[Beowulf] Re: Re: Home beowulf - NIC latencies
ashley at quadrics.com
Mon Feb 14 13:29:09 PST 2005
On 14 Feb 2005, at 18:58, Greg Lindahl wrote:
> On Mon, Feb 14, 2005 at 05:42:42PM +0000, Ashley Pittman wrote:
>> If you had a bunch of sends to do to N remote processes then I'd
>> you to post them in order (non-blocking) and wait for them all at the
>> end, the time taken to do this should be (base_latency + ( (N-1) * M
>> where M is the recpipiocal of the "issue rate". You can clearly see
>> here that even for small number of batched sends (even a 2d/3d nearest
>> neighbour matrix) the issue rate (that is how little CPU the send call
>> consumes) is at least as important that the raw latency.
> Unless I completely misunderstand your formula, M is not only the CPU
> the send call consumes. It's easy to find situations (fast cpu, slow
> network) where the cpu consumed isn't a part of M at all. Even for a
> modern 1 GByte/sec network, cpu consumed might not be a part of M.
I'm talking about our (Quadrics) network here which has a CPU offload,
a Wait call is simply a few function calls, a memory read (to test
completion), a mutex lock/unlock cycle and a linked list insertion,
nothing more. Some CPU is used in the send call as I said but outside
the two calls there is zero CPU usage although potentially reduced
memory to CPU bandwidth.
> Reducing CPU consumed can't hurt. But reasoning about it seems to be
> less useful than testing actual applications.
I do that as well.
More information about the Beowulf