Large Files on IA32

[Steffen Persvold]

Jakob Østergaard wrote:
> 
> On Fri, Jul 27, 2001 at 01:10:07PM -0500, Roger L. Smith wrote:
> > On Fri, 27 Jul 2001, Brian Macy wrote:
> >
> > >
> > > We have been trying to get support for large files (>2GB) working on
> > > our PentiumIII cluster running under RedHat 6.2 with a 2.2.x
> > > kernel.  In particular, most of our parallel programs are MPI
> > > programs and use MPI-IO (ROMIO) to do parallel I/O.  I'm interested
> > > if other people have had success with such a set up, and whether
> > > they'd be interested in sharing experiences.
> >
> > I have created files over 70GB in size on an xfs logical volume under
> > RedHat 7.1.  This doesn't really help you much with 6.2 or the 2.2 kernel,
> > I know, but I thought it was worth mentioning.  We've been using xfs on
> > our head node in production for several months (under quite heavy loads),
> > and haven't had any real problems with it.
> 
> Ok - just to clarify:
> 
> The 2GB limit is *not* within ext2.  If the filesystem is all you change, the
> 2GB limit will stay.
> 
> The limitation is in VFS, the virtual file-system layer on top of *any*
> filesystem you put in Linux.
> 
> The 2.4 kernels do not have this limitation anymore.  Files larger than 2 GB
> will work on 32-bit platforms on Linux 2.4 on any filesystem that supports it
> (including, of course, ext2).
> 
> However, you need more than just kernel support.  Your C library must be able
> to handle the bigger numbers as well.   And even with both kernel and C library
> in order, your applications and the libraries they use *also* must support
> this.
> 
> But don't worry - it's not so bad:  Many newer distributions, such as RH7.1,
> support this out of the box.  They have 2.4 kernels and glibc to go with it.
> 

Well, RedHat 6.2 with glibc 2.1.3 (stock) and 2.2.19-enterprise kernel have no problems
either.....

Regards,
-- 
  Steffen Persvold               Systems Engineer
  Email : mailto:sp at scali.no     Scali AS (http://www.scali.com)
  Tlf   : (+47) 22 62 89 50      Olaf Helsets vei 6
  Fax   : (+47) 22 62 89 51      N-0621 Oslo, Norway