[lug] Web crawler advice

[Nate Duehr]

karl horlen wrote:

>> Now move that webserver off-shore where it's harder to
>> get the attention 
>> of the authorities and/or the ISP... but keep your
>> ".com" domain name on 
>> the foreign IP address...
> 
> Wouldn't a domain registrar shut down this domain if contacted by "certain" authorities.  Not sure who those authorities would be or if they exist.  That seems logical.  It would prevent the site from wreaking havoc regardless of the country where the ISP resides.


Domain registrars aren't motivated to do so -- remember, they get paid 
by the domain.  They get nothing but goodwill for turning someone's 
domain off.  In general, registrars don't care.


>> You get the idea.  Evil incarnate.  And more common than
>> people think, 
>> sadly.  Indiscriminate web browsing and bad browser
>> behavior is right up 
>> there with some of the worst real "threats" to
>> modern computing as it 
>> gets.
> 
> So how would you really prevent this?  Unless you disable js in the browser, I don't think you'd be that successful.  It's hard to tell if a site is rogue until you actually visit it.
> 
> All of the major sites are moving to js (most use some sort of web 2.0 wizardry) these days and they don't degrade gracefully when disabled.  


JS has always been a hack to make web browsers into something they were 
never intended to be by their original design.  Obviously, the use of JS 
isn't going away, but it's only one example.

Flash, ActiveX, etc etc etc... all the same generic problem.  Browsers 
weren't supposed to be interactive terminals that could execute 
arbitrary code on your machine, but that's what they've morphed into.


Nate