1 prob. 1 question
Otis Gospodnetic (otisg@panther.middlebury.edu)
Tue, 26 Nov 1996 11:39:47 -0500 (EST)
Hi,
let me start with a weird problem. I am using the identical script that uses
libwww-perl 5.04 to fetch some html documents and use RobotUA so I can be more
polite.
I do everything the same as I would do with UserAgent module (except I give
the agent a name, emai, etc.) - I make headers....request a URL, etc.
The problem is now that on one machine this works as expected, but on the
other one the script keeps requesting /robots.txt (and I never ask it to do so
explicitly like I used to :) sicne I now know RobotUA handles that already)
Both machines have the same perl (5.003 something), LWP (5.04), same latest
libnet, both run AIX, but the 'good one' runs 3.2.5 while the 'bad one' has
4.1, but that shouldn't matter, should it ?
So, has anyone any idea what could be causing this ? Weird stuff....
Now a question - is it possible to have two RobotUA objects, running as
different UNIX processes running symultaneously and sharing the same Robot
Rules DBM without conflicting each other ? Or would it be better to run the
two objects from the same process but in different Threads (if perl allows
that ??) ?
Thanks !
Otis