Re: How to 'solve' 401 HTTP-errors?
=cw4t7abs (antiorp@tezcat.com)
Tue, 9 Feb 1999 13:44:47 -0600
>"Larry W. Virden" wrote:
>>
>> My first thought was that it might be _intentional_ - some people put
>> a lot of work into keeping web crawlers off their system...
>
>This was my first thought as well, and if that is the reason that I
>cannot request pages on these sites then I'll respect that.
>However, that still keeps me wondering how the server 'knows' that the
>page is NOT being requested by a Web browser, but rather by a robot...
>Does anybody have the answer to that question?
agent