[lug] Bad Apache problem?

[D. Stimits]

Justin-lists wrote:

> Skip to cliff notes at bottom if you don't want to read all the details.
>
> This is a followup to my problems over the weekend. History of events 
> is as follows: Power outage Wednesday from snow storm took server 
> down. Server came back up sometime Thursday and was working fine. I 
> started up2date -u Thursday night to get the glibc updates (Redhat 
> 8.0). I left that going overnight, Friday morning I could not ssh to 
> the server, although web/email seemed to be working ok. I was really 
> paranoid that I could not get into the box for seemingly no reason, so 
> I had the network cable pulled until I could get to it locally.
>
> Once I got to the box I was able to login at the console and things 
> looked ok. I did, however, notice this message in the messages log:
> Mar 21 00:11:30 oldschool init: Trying to re-exec init
>
> The timestamp on that error is sometime shortly after the glibc 
> updates were finished (I'm guessing) since I started the up2date 
> before I went to bed.
>
> I rebooted the server so I could watch the boot process and make sure 
> all was good. Server came up ok and looked good, I could login in 
> remotely again so that was a plus. Big problem came at this point. I 
> checked my website and all my hosted sites. I was getting a 403 Access 
> Denied error when trying to access anything hosted by the webserver. 
> The error in the apache error log read:
> [Sat Mar 22 17:18:52 2003] [crit] [client 193.252.111.200] 
> /home/httpd/.htaccess
> pcfg_openfile: unable to check htaccess file, ensure it is readable
>
> No .htaccess file existed in that directory and never has. I did a 
> touch /home/httpd/.htaccess and no longer got the access denied error 
> on my main site. However, the website was still not coming up fully 
> (missing images,etc). The error log then logged another message 
> identical to the above error except for a different directory (my 
> images directory). The server appeared to be checking *every* 
> directory under /home/httpd for a .htaccess file. If the file did not 
> exist the server was responding with access denied. Therefor resulting 
> in partially loaded sites.
>
> After a lot of troubleshooting and becoming extremely frustrated with 
> this problem, I found the issue to be related to the "AllowOverride" 
> feature in Apache. I set my AllowOverride for the doc root to "None" 
> and things all started working again. The .htaccess errors went away 
> and pages started loading fine. I did some tests and it seemed that 
> *any* setting other than None for the AllowOverride was causing the 
> same .htaccess messages noted above.
>
> The question/problem now is...Why is Apache checking for .htaccess 
> files in *every* directory and deny'ing access if they are not found 
> when AllowOverride is set to anything but None??? My Apache config has 
> not changed since March 5th and I have had my docroot AllowOverride 
> set to "Limit Authconfig FileInfo" for years without problems. I did 
> go and completely re-compile/install Apache again from the ground up, 
> still I get the same problem. Any ideas as to why Apache is behaving 
> this way? From what I can tell the problem arose after the first 
> reboot since installing the Redhat glibc patches.
>
> Cliff Notes:
> Apache giving 403 Access Denied when AllowOverride setting is set to 
> anything *but* None. Apache logs appear to be checking for .htaccess 
> files in *every* directory when AllowOverride is not set to None. If 
> no .htaccess file exists it gives access denied.

I would have to wonder which version of apache you had before, and which 
after? Are you sure that any updates of apache itself did not update 
config files? Conversely, are you sure that any update of apache was 
only a minor version update that does not update syntax of config files? 
If compiling your own, did you set it to use the same paths to find 
config files?

If updating library files that apache links to, perhaps changing those 
link files causes it to behave differently (especially with regard to 
plugins to apache itself). If you run ldd against the apache binary 
(whether named "apache" or "httpd"), did any of the files it links 
against change? If one of those changed, is it a minor version change, 
or a major version change?

Now I'm not big on apache config, but I would have to also interpret the 
AllowOverride None to mean that there is no purpose in checking for a 
.htaccess, while allowing any form of override would mean that it then 
must check for .htaccess in any directory this is available to...the 
allow override can be specified for everything, or for only particular 
directories. If you allow override and don't provide a .htaccess file 
for every relevant directory, it's probably just good security that it 
complains; it'd just make sense that if .htaccess is missing and it has 
been specified, that the directory should be closed down until a 
.htaccess comes back. Exactly how was AllowOverride specified...was it 
for everything, or specified for a specific location? What you're seeing 
may just be a reflection of saner security defaults compared to older 
versions.

D. Stimits, stimits AT attbi DOT com