[Beowulf] malloc on filesystem

[Serissa]

If you are willing to mmap the whole file, then dlmalloc can do this.  The issue is that it expects the storage pool to be accessible with pointers.
I am not aware of an allocator that uses function calls to read and write its own metadata (so that you can abstract the metadata), but if you find one I'd like the reference!  In my previous job we wanted such a thing so that host code could run an allocator for GPU memory without load store access to the memory being allocated, and we couldn't find one. There were too many places in dlmalloc that did pointer access so it was too hard to figure out which ones were accessing the storage pool and which were local.

-Larry

> On Feb 5, 2025, at 9:33 AM, Michael DiDomenico <mdidomenico4 at gmail.com> wrote:
> 
> this might sound like a bit of an oddify, but does anyone know if
> there's a library out there that will let me override malloc calls to
> memory and direct them to a filesystem instead?  ie using the
> filesystem as memory instead of ram for a program.  ideally something
> i can LD_PRELOAD on top of a static binary.  understandably this is
> generally a silly thing to do, but you know, users...
> 
> google is failing me, my search terms likely aren't right.  i'm
> looking through some of the older checkpointing codes at the moment.
> maybe someone cut shortcut my search
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf