[lug] Httpd log question

[Alan Robertson]

Lee Woodworth wrote:
> Bill Thoen wrote:
>> I'm running RH9 with apache and as I was looking through my 
>> /var/log/httpd/access_log file I noticed a lot of web "hits" aren't 
>> showing the referring URL. For example, I see things like this:
>>
>> 69.242.25.101 - - [28/Aug/2005:06:54:23 -0600] "GET 
>> /notebook/unusual.htm HTTP/1.1" 200 8497 "-" "Mozilla/4.0 (compatible; 
>> MSIE 6.0; Windows NT 5.0)"
>>
>> If the person was referred to the web page from a link or a search 
>> engine,
>> the "-" after the number 8497 above will contain the URL of the web page
>> they jumped in from. I figured out that one way that the referring URL
>> can show up as "-" is if the person manually types the URL into their
>> browser, but it seems that for this "unusual.htm" page I get quite a few
>> like this. My other pages don't seem to show this behavior as much.
>>
>> Does anyone here know what other ways a web surfer can get to a page and
>> leave no trace as to where they came from? I'm sort of curious because 
>> it's my second-most popular page and I'd like to figure out what I'm 
>> doing right.
> Web spiders or web browsers that are watching a site. The referer (sic)
> request header is optional. Programs may or may not supply it. They can
> lie too. I haven't checked, but it may be that opening a link in a new
> tab with firefox doesn't send the referer header. Use the host command
> on some of the ips to get an idea of the requestor's domain. It will
> also help you figure out if some of them are search engines.

Also, MSIE can "bottle up" web pages for later surfing.  Maybe someone 
is playing with that feature using your site as his/her example.

Bookmarks also show "-" as the referrer.


-- 
     Alan Robertson <alanr at unix.sh>

"Openness is the foundation and preservative of friendship...  Let me 
claim from you at all times your undisguised opinions." - William 
Wilberforce