Re: Timeout Problems with Parallel::UserAgent
Tom Scanlan (tom@squarefish.com)
Mon, 21 Jun 1999 06:59:59 -0400
Well, I don't think the timeout problem we are having is a bug with
LWP::Parallel. Netscape has the same problem, as does the linear
UserAgent, and Lynx. I'm not exactly sure if all cases are like the
following (and I'm no network specialist), but my problem pops up when I
try to connect to either a a bad domain name or a screwy server.
I'll use the site I'm trying to load now: jrdfarms.com, although there
are many such domains. A whois says they are around and reachable:
[rs.internic.net]
Registrant:
JRD FARMS (JRDFARMS2-DOM)
PO Box 1410
Vernal, UT 84078
US
Domain Name: JRDFARMS.COM
Administrative Contact, Technical Contact, Zone Contact:
BHCOM HOSTMASTER (HC391-ORG) hostmaster@BHCOM.COM
760-360-4600
Fax- 760-772-3654
Billing Contact:
Dirzuweit, John R (JRD136) jrdfarms@BIGFOOT.COM
435-789-4513
Record last updated on 03-Feb-99.
Record created on 03-Feb-99.
Database last updated on 20-Jun-99 08:12:04 EDT.
Domain servers in listed order:
NS1.4SERVERS.COM 209.176.24.140
NS2.4SERVERS.COM 209.176.24.150
You agree that you will not reproduce, sell, transfer, or
modify any of the data presented in response to your search request, or
use of any such data for commercial purpose, without the prior
express written permission of Network Solutions.
but trying to load the website for http://jrdfarms.com takes and
extreamly long time (more than 8 mins now). Netscape finally tells me
that the server cannot be found. I tried to get to the site using a
little non-parallel RobotUA from lwp v5.41:
#!/usr/bin/perl
use lib '/home/squarefish/bin/perl/';
use LWP::RobotUA;
use HTTP::Request;
use HTTP::Response;
use URI::URL;
my $url = new URI::URL('http://jrdfarms.com/');
print "uri as_string: " . $url->as_string ."\n";
my $spider = new LWP::RobotUA('squareSpider v0.2',
'tom@squarefish.com');
$spider->delay(.2);
my $request = new HTTP::Request('GET', $url);
my $response = $spider->request($request);
print "request as string: " . $request->as_string . "\n";
print "header: ". $response->header . "\n";
print "message: ". $response->message . "\n";
print "content: " . $response->content . "\n";
print "as string: " . $response->as_string . "\n";
and got the following output:
request as string: GET http://jrdfarms.com/
From: tom@squarefish.com
User-Agent: squareSpider v0.2
header:
message: Can't connect to jrdfarms.com:80 (Connection timed out)
content:
as string: 500 (Internal Server Error) Can't connect to jrdfarms.com:80
(Connection timed out)
Client-Date: Mon, 21 Jun 1999 10:37:00 GMT
The server is unreachable, possibly a DNS problem (?), or the server is
playing dead (?). Trying to connect to the domain servers 209.176.24.140
and 209.176.24.150 tells me that they are busy or refusing connections.
The question I have is why do we wait so long for an answer? I haven't
figured out if this is a problem with the socket or something at a
higher level. The timeout doesn't seem to have any affect on how long
we wait to find the server, only how long to wait when no data is being
sent of the connection. Does anyone know of a work around to waiting
such a long time for a reply in this case?
--
O
o o
Tom Scanlan /\/ tom@squarefish.com
4750 Hartland Pkwy. Suite 210, \/\ squareFish Media Services, Inc.
Lexington, KY. 40515 Phone:[606.245.9656] web design. web hosting.