[Beowulf] Parallel memory

[Robert G. Brown]

On Tue, 18 Oct 2005, Henderson, T Todd @ IS wrote:

> Are there any drivers, tools, etc. that can make the memory space on a
> cluster look shared, something like pvfs but for memory?  I'm sure there
> would be a speed hit, but in this instance, speed isn't the problem as much
> as memory.  We have a code that uses a huge amount of memory and the memory
> usage is proportional to the cube of the problem size, but the time for it
> to run isn't too much of an issue.  I've been asked to parallelize the code
> using mpi which is going to be a major effort.  However, I thought that if
> there was anyway, even if inefficient speed wise, to create a virtual
> parallel memory system it would be better than it using swap space and save
> a bunch of coding time.

There have been projects like this in the past; I don't know what their
current status is.  The Trapeze project at Duke (Jeff Chase) is pretty
much exactly this -- creating a large VM (something like a CC-NUMA if I
recall) using cluster nodes and myrinet for their MEMORY (and myrinet's
relatively low latency for "memory-like" retrieval).  I think that the
project is no longer active, but chase at cs.duke.edu (or maybe his student
Justing Moore who frequents this list) can update you on it.

There was once also an online list discussion about swapping to an NFS
mounted remote exported ramdisk, IIRC.  In principle all of that should
work although I have NOT tried it and cannot say for certain.  Making a
big ramdisk is fairly straightforward.  Once made and mounted, it can be
NFS exported.  The trick then is whether it indeed possible to swap to
an NFS mounted swapspace.  On Suns this used to be routine (otherwise
e.g. the SLC and ELC, fully diskless sparc nodes from the 90's, wouldn't
have worked).  On linux back in the 2.0 and maybe 2.2 series, I think
that there was a problem -- maybe with page sizes? -- and it wouldn't
work.  But I THINK that I recall hearing that remote linux swap now
works, and if so that might be a way to go to get to a largish VM.

I'd really be interested (I'm sure the whole list would be) if you try
the latter and it works, especially if it works pretty "well" (e.g.
orders of magnitude faster than disk swap if not as fast as real
local memory).  It would also give a great idea to all those cs grad
students looking for a useful project, as very large VM systems are
almost as advantageous to certain kinds of research as very FAST local
memory systems are to others.

> Also, are there any tools to help implement mpi in an older code?

I've decided that I'm too ignorant to help on MPI questions, so I'll
leave this to maybe Jeff Squyers and other people who are real experts;-)

    rgb

>
>
>
> Any thoughts?
>
>
>
> Thanks,
>
> Todd
>
>
>
>

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu