[lug] Httpd log question

[Lee Woodworth]

Bill Thoen wrote:
> I'm running RH9 with apache and as I was looking through my 
> /var/log/httpd/access_log file I noticed a lot of web "hits" aren't 
> showing the referring URL. For example, I see things like this:
> 
> 69.242.25.101 - - [28/Aug/2005:06:54:23 -0600] "GET /notebook/unusual.htm 
> HTTP/1.1" 200 8497 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 
> 5.0)"
> 
> If the person was referred to the web page from a link or a search engine,
> the "-" after the number 8497 above will contain the URL of the web page
> they jumped in from. I figured out that one way that the referring URL
> can show up as "-" is if the person manually types the URL into their
> browser, but it seems that for this "unusual.htm" page I get quite a few
> like this. My other pages don't seem to show this behavior as much.
> 
> Does anyone here know what other ways a web surfer can get to a page and
> leave no trace as to where they came from? I'm sort of curious because 
> it's my second-most popular page and I'd like to figure out what I'm doing 
> right.
Web spiders or web browsers that are watching a site. The referer (sic)
request header is optional. Programs may or may not supply it. They can
lie too. I haven't checked, but it may be that opening a link in a new
tab with firefox doesn't send the referer header. Use the host command
on some of the ips to get an idea of the requestor's domain. It will
also help you figure out if some of them are search engines.

> 
> - Bill Thoen
> 
> 
> _______________________________________________
> Web Page:  http://lug.boulder.co.us
> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
> Join us on IRC: lug.boulder.co.us port=6667 channel=#colug