[lug] R00tKIT!! Raah!

[Bear Giles]

Michael Belanger wrote:
> Now, here is a question, can the 'apache' user install a rootkit if they 
> are not root?

Our last two compromises were thru apache.  One was due to an old 
version of mod_ssl.  The other was due to an old version of 
awstats.  (The former because we couldn't update an ancient 
version of RH, the latter because the Debian maintainer didn't 
realize he needed to issue a critical security update.)

> I fear I may need to travel out there to rebuild the server... Anyone 
> know if it is possible to 'clean' the system?

The "correct" answer is no.

The "practical" answer is that there's a very real chance that the 
attacker was a script kiddie who didn't exploit his success.  The 
bigger problem in this case is that you may have been hit by 
several attackers through the same exploit and that complicates 
the cleanup.

It comes down to an informed gamble.  How much will you lose if 
you guess wrong and can't clean out the damage, how much will you 
lose from the effort required to do a clean reinstall?  Don't 
forget that you have to assume that your backups are compromised. 
  That's not an issue if it's just data (e.g., mailboxes and 
static web pages), more complicated if you have third-party 
software or a lot of local programs and scripts.  (BTW this is 
another argument for separate partitions for server data.  They 
can be backed up and restored without too much concern and mounted 
'noexec'.)

If you do decide to clean the system, a few good places to start 
(after reinstalling a clean kernel) is to look for executable 
files in /dev, /tmp and /var/tmp, check all files with the SUID 
and SGID bit set and force a reinstall of procps, passwd, login 
and sudo.  And netstat.  I'm sure others can add to that list.

Speaking of netstat - get a list of every process that's listening 
(netstat -l, iirc) and then use lsof to identify the process 
that's attached to that port.  Make sure you understand why it's 
there.

Again, you're still hosed if you had a competent attacker.  On the 
other hand if it was a worm it may be better to spend a few hours 
cleaning out the system than spending a few days to rebuild it.

Bear