[lug] More Server Problems

[George Sexton]

My server tanked again (reference):

http://archive.lug.boulder.co.us/Week-of-Mon-20060123/031529.html

Since that thread I upgraded to 2.6.14.6 kernel version and I'm still having
the same issue. After the server crashes (message unknown but I'm guessing
its ReiserFS file system corruption related) I have to run reiserfsck and
use the --rebuild-tree option.

I ran memtest86 version 3.2 through 4 complete cycles and found no memory
issues. I also checked the hardware monitoring from the bios. It looks like
the temperature is well within reason for CPU and motherboard (31-37 C).

I haven't had a chance to take the disks offline but there are no kernel
messages normally associated with failing disks (write timeouts, dma
timeouts, etc). I'm really suspecting a kernel issue in either the md or
reiserfs subsystem. I suppose it could be a more generic issue relating to a
Pentium D CPU but I think I would get some sort of message and the reiser
file system wouldn't be hosed.

To avoid having to drive to the COLO facility, I've hooked up the machine to
another one using a null modem cable so I can do a serial console. The BIOS
supports a serial console, and I configured GRUB and the kernel to use a
serial console, and also uncommented an entry in the inittab file to start a
serial console on ttyS0. 

I now have a serial terminal session to the affected computer. I'm running
Minicom and I've logged in as root and set minicom to capture data from the
session.

The one thing I'd like to sort out how to do is to make SYSLOG-NG echo any
kernel or other error messages to the console running on ttyS0.

Does anyone know a simple way of doing this? I'd also be interested in ideas
on how I can fix this issue. The only solution that I can think of would be
to re-install linux, and choose a 32-bit installation. I have a machine that
has the same disk sub-system but is running 32 bit kernel/linux and I'm not
having any issues.


George Sexton
MH Software, Inc.
http://www.mhsoftware.com/
Voice: 303 438 9585