[lug] Huge stack size--any reason to avoid?

[bgiles at coyotesong.com]

Is stack space swappable?  I'm not saying that you would want to run other
processes, but you would certainly want to have that option.

As for the question in general, my approach is to create a mmap'd buffer
that's backed by the filesystem.  It should normally stay in memory, but
could be swapped to disk (without consuming swap space!) and gives you an
easy way for another process to examine the contents.

I don't have the man(2) pages handy, but it's something like:

  int fd = fopen("file", O_CREAT|O_RDWR|...)
  flock(fd,...)
  ftrunc(fd,...)
  char *ptr = mmap(...)
  // use data pointer...

That is, create the file, lock it, 'truncate' it to some ridiculous size
(which will make it a sparse file), then mmap it.  You can unlink the file
immediately after it's created, if you're worried about unused files
hanging around.  Other applications can also peek into the file since
buffer blocks are just buffer blocks.

You can then use the 'ptr' as any other buffer.  You could run a memory
allocator over it, or just use one file per very large object.  You could
even overload 'new' and 'delete' so they use this buffer!

This isn't as easy to use as the stack and heap, but it's a lot more
flexible and lets you handle objects far larger than your physical memory
+ swap.