Re: "100 levels deep in subroutine calls" with multiple GETs and LWP::UserAgent

• Messages sorted by: [ date ][ thread ][ subject ][ author ]
• Next message: Shane Robinett: "Newbie"
• Previous message: Rolf Nelson: "HTML::FormatText list-handling extension"

Gisle Aas wrote:
> 
> In message <3234E188.10341F7A@sifry.com>, "David L. Sifry" writes:
> > Here's the program I wrote that fails on iteration 44:
> 
> The program looks ok to me.  I see no problem when running your
> program on my machine.

I did even more checking over the last 2 days, and I think I know more
about the problem, but I still don't have a solution.

Please, somebody help me!!!

I believe the problem is cropping up during a die in one of the evals in
LWP::UserAgent.pm.  Line 231 does this:

eval {
$response = $protocol->request($request, $proxy,
			       $arg, $size, $timeout);
};

My guess is that subroutines are called and are not popped off the
stack. In the case of a bad URL address, subroutines called during the
eval and before the exception is thrown are NOT popped off of the
stack.  Therefore, as the program iterates, garbage accululates on the
stack, finally forcing perl to segmenation fault when the stack reaches
100.

My evidence for this theory is twofold: First off, 

When I give the test program a bad URL (like: http://badhost/), I notice
that the eval in LWP:UserAgent goes down 3 subroutines before die-ing
and getting trapped by the eval.  I also know that I can do 30
iterations of my test program with a bad address in the URL.  Lastly,
there are 10 subroutines on the stack when it crashes.(found using the T
command in the perl debugger.)

(30 * 3) + 10 = 100.  

When I give my test program a legitimate URL, it pulls down 45 copies of
the URL, and then fails. When it fails, there are 10 subroutines on the
stack.  I can't find what's die-ing here, but if you look at the math:

(45 * x) +10 = 100;   x = 2! (No fractions!)

it would lead me to believe that after each call to
HTTP::Request->new(GET=> $url); that is successful, 2 subroutines are
added to the stack and not removed.

I'm tearing my hair out over this.  I can't find the code to change to
correct this, or even verify for sure if my theory is correct.  I get
this to happen each and every time I run this code, but yet Gisle tells
me that he couldn't reproduce it on his machine.  I remade perl 5.003
from scratch, and then re-made each of the necessary modules for
libwww-perl: IO-1.09, MD5-1.7, libnet-1.01, libwww-perl-5.01 and also
libwww-perl-5.02, but I still get this crash!  Could this be a perl
bug?  A linux bug?  Could you please run this little program and tell me
if you get a segmentation fault or if it runs cleanly on your machine?

I'm running a linux 2.0.6 system, perl 5.003, libdl.so.1.8.0,
libm.so.5.0.6, libc.so.5.2.18.

Here's my test program:

------------------------------------------------------------------
#!/usr/bin/perl -w
use strict;

use LWP::Debug qw(level);
level('+');

use LWP::UserAgent;
my $ua = new LWP::UserAgent;
$ua->agent("test/0.1");
$ua->from('david@sifry.com');

my $url = "http://badhost/";
my $findurl = HTTP::Request->new(GET => $url);

my $i;
for ($i=0;$i<50;$i++) {
    $ua->request($findurl);
    print "GET: ", $i, " " , $findurl->url,"\n";
}
------------------------------------------------------------------

Thanks so much in advance for your help!  I really appreciate it!
-- 
Dave Sifry
david@sifry.com, sifry@aptltd.com
(408) 471-0667 (voice)
(408) 471-0666 (fax)

• Next message: Shane Robinett: "Newbie"
• Previous message: Rolf Nelson: "HTML::FormatText list-handling extension"