[lug] C program freezing machine

[Michael J. Hammel]

On Tue, 2008-04-15 at 12:24 -0400, gordongoldin at aim.com wrote:
> I have a programmer with big number cruncher (probably memory
> intensive) programs.
> He submits his program - the machine freezes.
> What are the best items to look at first in trying to track down what
> could be happening.

User level applications don't generally lock the entire system, though
they can lock the UI.  Can you still ping the machine when this program
runs and appears to lock the system?  If you can, then the kernel is
still buzzing along and the system isn't completely locked even if user
space is.

On Unix/Linux, for the entire system to freeze because of an application
program the program must be causing either a spin loop in the kernel
(which I would imagine is a bug in the kernel or driver), a memory
overwrite in the kernel (also a bug) or a synchronous request in the
kernel to never be acknowledged (mostly a bug except for swap space
problems, I suspect).  One possibility is his program absorbs so much
memory and there is so little swap space that when it comes time for his
application to be swapped out of memory to let other applications run,
it can't be swapped.  I'm not sure if this will completely lock up Linux
or not, however, though I can imagine it would lock up the UI.  I was
pretty sure the kernel knew how to handle this situation so that at
least the kernel would keep running.  But I could be wrong about that.

My suggestion is to write the program so it allocates a block of memory
and then manages that memory itself with it's own swap space.  This is
similar to how the GIMP deals with very large images, using a tile-based
systen.  There is no reason for a single program to completely absorb
all of memory with continuous calls to malloc() (though ulimits should
probably prevent it from being able to do that anyway).  If it's that
big a hog you probably didn't want it running under a multiuser OS
anyway.  It would probably be happier running under DOS (honestly).
-- 
Michael J. Hammel                                    Principal Software Engineer
mjhammel at graphics-muse.org                           http://graphics-muse.org
------------------------------------------------------------------------------
Share your knowledge. It's a way to achieve immortality.
  --  Credited to the Dalai Lama.