[Beowulf] Parallel memory
rw26 at wright.es.its.nyu.edu
Tue Oct 18 18:44:32 PDT 2005
At cluster 2005 in Boston last month, I listened to a prsentation about
swapping over infinband's rDMA to remote memory. They had a latge
quicksort running at 1.7 times the speed of doing it in local memory only.
On Tue, 18 Oct 2005, Robert G. Brown wrote:
> On Tue, 18 Oct 2005, Henderson, T Todd @ IS wrote:
> > Are there any drivers, tools, etc. that can make the memory space on a
> > cluster look shared, something like pvfs but for memory? I'm sure there
> > would be a speed hit, but in this instance, speed isn't the problem as much
> > as memory. We have a code that uses a huge amount of memory and the memory
> > usage is proportional to the cube of the problem size, but the time for it
> > to run isn't too much of an issue. I've been asked to parallelize the code
> > using mpi which is going to be a major effort. However, I thought that if
> > there was anyway, even if inefficient speed wise, to create a virtual
> > parallel memory system it would be better than it using swap space and save
> > a bunch of coding time.
> There have been projects like this in the past; I don't know what their
> current status is. The Trapeze project at Duke (Jeff Chase) is pretty
> much exactly this -- creating a large VM (something like a CC-NUMA if I
> recall) using cluster nodes and myrinet for their MEMORY (and myrinet's
> relatively low latency for "memory-like" retrieval). I think that the
> project is no longer active, but chase at cs.duke.edu (or maybe his student
> Justing Moore who frequents this list) can update you on it.
> There was once also an online list discussion about swapping to an NFS
> mounted remote exported ramdisk, IIRC. In principle all of that should
> work although I have NOT tried it and cannot say for certain. Making a
> big ramdisk is fairly straightforward. Once made and mounted, it can be
> NFS exported. The trick then is whether it indeed possible to swap to
> an NFS mounted swapspace. On Suns this used to be routine (otherwise
> e.g. the SLC and ELC, fully diskless sparc nodes from the 90's, wouldn't
> have worked). On linux back in the 2.0 and maybe 2.2 series, I think
> that there was a problem -- maybe with page sizes? -- and it wouldn't
> work. But I THINK that I recall hearing that remote linux swap now
> works, and if so that might be a way to go to get to a largish VM.
> I'd really be interested (I'm sure the whole list would be) if you try
> the latter and it works, especially if it works pretty "well" (e.g.
> orders of magnitude faster than disk swap if not as fast as real
> local memory). It would also give a great idea to all those cs grad
> students looking for a useful project, as very large VM systems are
> almost as advantageous to certain kinds of research as very FAST local
> memory systems are to others.
> > Also, are there any tools to help implement mpi in an older code?
> I've decided that I'm too ignorant to help on MPI questions, so I'll
> leave this to maybe Jeff Squyers and other people who are real experts;-)
> > Any thoughts?
> > Thanks,
> > Todd
Senior Systems Administrator
New York University
More information about the Beowulf