Memory Limits on Linux on x86

[Donald Becker]

On Fri, 19 Jul 2002, Mark Hahn wrote:

> Subject: Re: Memory Limits on Linux on x86
> > Yes, the patch just sets the boundary between kernel and user space.
> > You can change it to an arbitrary split, but I believe it's still
> > checked in some places using a binary mask.
> > 
> > See  __PAGE_OFFSET  in <asm/page.h>
> 
> so the split is arbitrary, for some limited subset of arbitrary values ;)

Having a named constant makes it simple matter of scanning and fixing the
usage.
A quick scan through the kernel shows that there are still a few
places where that should be checked.  The usage difference is
   adding with "|" (logical OR) instead of "+", and
   comparing using "&" instead of ">=".
Using these bit operation used to be slightly faster, but there is now
performance difference.

> as Don points out, all of this applies only to modern kernels, not 2.2...

But 2.2 is still a modern kernel!

> > But practically you only care about the single block I/O device you are
> > using: the IDE or SCSI controller and the bus bridges it talks through
> > must support 64b initiator addresses or you will be doing massive
> > copying.
> 
> but aren't there other masters that you'd really like to be 64b-capable?  
> like, as a *completely* random example, a NIC?

You would think I would be out there pitching the latest and greatest,
just like the video card guys that think that this months hottest video
card is the only one worth supporting.  But I'm a very systems-oriented
person, and don't want to add complexity unless it is necessary.

Ethernet NICs have few reasons to use 64 bit mode.  It's easy enough to
allocate all network buffers below 4GB, and there are good performance
reasons to do so even with 64b-capable cards. 

The only case where you can't easily insure that network buffers are below
4GB is when using the current tx-zero-copy scheme.  I don't see that as a
problem because I don't think the current approach is a performance win.

The performance benefit from zero copy will come only when the NIC
handles the TCP segmentation.  Zero copy  Tx should only be used when
    the NIC supports 64b addressing and full TCP Tx offload,
    the TCP window is 64KB+,
    the block to be transmitted is 32KB+ in contiguous pages, and
    the transmitter is not congested.
A smaller size results in the overhead of locking pages on a SMP
overwhelming the benefit of avoiding a copy.

I see much more potential in an iSCSI enabled gigabit NIC with a design
that looks like a traditional NIC and a traditional SCSI controller
mixing traffic to the transceiver.

-- 
Donald Becker				becker at scyld.com
Scyld Computing Corporation		http://www.scyld.com
410 Severn Ave. Suite 210		Second Generation Beowulf Clusters
Annapolis MD 21403			410-990-9993