[lug] Wget Behavior Differnces on Different Machines

Bill Thoen bthoen at gisnet.com
Tue Nov 8 14:55:02 MST 2005


Evelyn Mitchell wrote:

>I don't know why it's failing one place and not the other. I can't see that
>either session you've included is succeeding.
>
>Try a smaller chunk of data, like for a specific state.
>  
>
The place it's hanging on is the line:

Connecting to catalog.mso.census.gov[148.129.75.139]:443...


Which occurs on one machine but not on the others. For now I've got 
around the problem by trying the same wget command on another machine, 
and it works perfectly. The only thing I can guess at now is that on the 
machine where it doesn't work, wget is an old version -- 1.8 or so, 
where on the machines where it succeeds the version is 1.10. But I'm 
still curious why in that one case it wanted to access a secure server 
for something , but not in the other cases. The other machines never 
access this secure connection (or at least never log it) but this 
machine does for some reason. And I used the exact same wget 
command-line switches in all cases.

However, the problem is no longer a critical one for me now... just a 
head scratcher...

>Evelyn
>
>* On 2005-11-08 12:00 Bill Thoen <bthoen at gisnet.com> wrote:
>  
>
>>Hi folks,
>>I'm still muddling about with wget trying to download all the Census 
>>Bureau's TIGER data fom their server at 
>>http://www2.census.gov/geo/tiger/tiger2004se. When I tried it on one of 
>>my machines everything worked great until I ran out of free disk space 
>>5GB later. So now I'm trying it on a friend's machine (where there's 
>>lots of space) but as soon as the wget process gets startred it stalls 
>>trying to connect to a secure connection for some reason. On my machine 
>>it sails right past this step. If anyone knows what's wrong, I'd like to 
>>know what I need to do to get wget to work.
>>
>>Here's thg wget command I'm using:
>>wget -r -l2 -nH http://www2.census.gov/geo/tiger/tiger2004se
>>
>>Below is what I see in the logs from the runs on the two machines, On 
>>machine 'A' here's the last thing I see before it just hangs  (the log 
>>from my machine follows):
>>---
>>Loading robots.txt; please ignore errors.
>>--11:27:54--  http://www2.census.gov/robots.txt
>>          => `robots.txt'
>>Connecting to www2.census.gov[148.129.75.27]:80... connected.
>>HTTP request sent, awaiting response... 200 OK
>>Length: 0 [text/plain]
>>
>>   [ <=>                                                  ] 
>>0             --.--K/s
>>
>>11:27:54 (0.00 B/s) - `robots.txt' saved [0/0]
>>
>>Loading robots.txt; please ignore errors.
>>--11:27:54--  https://catalog.mso.census.gov/robots.txt
>>          => `robots.txt'
>>Resolving catalog.mso.census.gov... done.
>>Connecting to catalog.mso.census.gov[148.129.75.139]:443...
>>
>>--- at this point, this process just hangs waiting for Godot or something.
>>And here's the same step from my machine, where it works fine, going 
>>past this point and continuing to download files (but runs out of space 
>>eventually.)
>>---
>>
>>Loading robots.txt; please ignore errors.
>>--11:02:00--  http://www2.census.gov/robots.txt
>>          => `robots.txt'
>>Connecting to www2.census.gov|148.129.75.27|:80... connected.
>>HTTP request sent, awaiting response... 200 OK
>>Length: 0 [text/plain]
>>
>>   0K                                                           0.00 B/s
>>
>>11:02:00 (0.00 B/s) - `robots.txt' saved [0/0]
>>
>>--11:02:00--  http://www2.census.gov/icons/blank.gif
>>          => `icons/blank.gif'
>>Connecting to www2.census.gov|148.129.75.27|:80... connected.
>>HTTP request sent, awaiting response... 200 OK
>>Length: 148 [image/gif]
>>
>>
>>_______________________________________________
>>Web Page:  http://lug.boulder.co.us
>>Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>>Join us on IRC: lug.boulder.co.us port=6667 channel=#colug
>>    
>>
>This email is: [ ] actionable [x] fyi [ ] social
>Response needed: [ ] yes [x] up to you [ ] no 
>Time-sensitive: [ ] immediate [x] soon [ ] none
>
>
>  
>




More information about the LUG mailing list