Re: How to 'solve' 401 HTTP-errors?

Bjorn Hermans (bjorn.hermans@capgemini.nl)
Tue, 09 Feb 1999 08:41:58 +0100


"Larry W. Virden" wrote:
> 
> My first thought was that it might be _intentional_ - some people put
> a lot of work into keeping web crawlers off their system...

This was my first thought as well, and if that is the reason that I
cannot request pages on these sites then I'll respect that.
However, that still keeps me wondering how the server 'knows' that the
page is NOT being requested by a Web browser, but rather by a robot...
Does anybody have the answer to that question?


Greetings,
Bjorn Hermans.