Locality and caching in parallel/distributed file systems

[Eray Ozkural]

On Tuesday 03 December 2002 04:57 pm, Andrew Fant wrote:
> Morning all,
> 	Lately, I have been thinking a lot about parallel filesystems in
> the most un-rigourous way possible.  Knowing that PVFS simply stripes the
> data across the participating filesystems, I was wondering if anyone had
> tried to apply caching technology and file migration capacities to a
> parallel/distributed filesytem in a manner analagous to SGI's ccNuma
> memory architecture.  That is, distributing files in the FS to various
> nodes, keeping track of where the accesses are coming from, and moving
> the file to another node if that is where some suitable percentage of the
> reads and/or writes are coming from.  Also, potentially allowing blocks
> from local files to be cached in disk on a local node until a write to
> those blocks elsewhere invalidates the cache (I know the semantics for
> this theoretically are in NFS, but NFS doesn't scale, and is dead 8-).

In particular, this problem seems to be similar to a problem in Information 
Retrieval which we had given some thought to. It is likely to have a 
combinatorial solution since we have many nodes and many files :>

I must meditate further.

Thanks,

-- 
Eray Ozkural <eozk at bicom-inc.com>
Software Engineer, BICOM Inc.
GPG public key fingerprint: 360C 852F 88B0 A745 F31B  EA0F 7C07 AE16 874D 539C