Re: LWP UserAgent Cookie Problem - redirect to same URI (?)
Steve Borruso (sborruso@stny.rr.com)
Sat, 17 Mar 2001 10:38:04 -0500
Erik and Randal,
Thank you VERY much for your time, patience, guidance, (and code !) in helping
me better understand and solve the problem I was having. I've implemented a
routine based on the code you've sent and all is right with the world ! (at least
for
this brief moment in time :)
Thanks again !
Steve Borruso
"Randal L. Schwartz" wrote:
> >>>>> "Kingpin" == Kingpin <mthurn@tasc.com> writes:
>
> Kingpin> You must be kidding when you say you've been working on this for a
> Kingpin> week, right? It only took me 20 minutes to realize the problem AND
> Kingpin> come up with this solution when I wrote my spider:
>
> Kingpin> Don't try to get LWP to do it automatically!!!!!! Tell LWP not to do
> Kingpin> any redirects, then write your own loop to parse and follow them
> Kingpin> yourself (adding cookies or passwords or whatever each time). Stop
> Kingpin> the loop when you don't get another redirect, or when get to the page
> Kingpin> you want (or bail out after 5 iterations to avoid infinite loop).
>
> Here's a snippet I wrote a few days ago to log in my "virtual browser"
> to www.imdb.com, which needs both cookies and a login form:
>
> #!/usr/bin/perl
> use strict;
> $|++;
>
> use LWP::UserAgent;
> use HTTP::Cookies;
> use HTTP::Request::Common;
>
> my $ua = LWP::UserAgent->new;
> $ua->cookie_jar(HTTP::Cookies->new(file => 'cookie_jar', autosave => 1));
>
> my $r = $ua->simple_request(POST "http://us.imdb.com/register/login",
> {
> url => '',
> login => 'my_imdb_email@stonehenge.comm',
> passwd => 'my_sekret_password',
> 'Login!' => 'Login!',
> });
> while ($r->is_redirect) {
> my $u = $r->header('location') or die "missing location: ", $r->as_string;
> print "redirecting to $u\n";
> $r = $ua->simple_request(GET $u);
> }
> print $r->as_string;
>
> LWP::UserAgent handles all cookie management for me, once given a
> cookie. I wrote the is_redirect loop because IMDB wants a POST
> to redirect to a GET. Evil. :(
>
> (Yes, this the beginnings of an upcoming column... :)
>
> --
> Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
> <merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
> Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
> See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!