Re: handling redirects with cookies

Jeff W. Sondeen (sondeen@almaak.usc.edu)
Wed, 06 Jan 1999 16:31:28 -0800 

It seems that this thread about 'handling redirects with cookies'
(well describe by Maurice Aubrey) has not been resolved, regarding the
problem that when you ask for page X, and receive a redirect to the
same page X but including a bunch of new cookies to attach to followup
request headers, the LWP UserAgent.pm script dies with 'Redirect loop
detected'.

has anyone got a fix? 

thanks,

/jeff



Here is a short history of this problem:

Maurice Aubrey (maurice@hevanet.com)
Thu, 16 Jul 1998 16:24:12 -0600 

     Messages sorted by: [ date ][ thread ][ subject ][ author ] 
     Next message: Benjamin M. Hill: "Re: Sitemapper 1.003" 
     Previous message: Ave Wrigley: "Sitemapper 1.003" 



I'm experiencing a problem when trying to fetch a page
from a site which employs cookies.

The site is returning a 302 response, with a redirect
to the same exact URL.  It also includes a cookie.  
LWP::UserAgent notices that it's the same URL, and refuses to 
follow it (including the "Client-Warning: Redirect loop detected"
message instead).  

The site is returning 302, because it wants you to fetch the
same page, but wants you to include the cookie the second time
around (which is what terminates the redirects).  LWP doesn't
seem to do that.

Can we improve this?  I'm wondering if, once the redirect is
noticed and the request is cloned, the response object could
be scanned for cookie headers, and if present, add a cookie
jar to the new request?  It would also note that cookies were
set, and would agree to follow the redirect even if it is
identical (and fail only if max redirects is reached).

We'd also need to make sure that the cookie-set headers are
included in the final response object.

Would this be a useful change, or is it complicating things too much?
Perhaps I should just code my app to look for redirects and handle
the cookie stuff myself?

I'm using libwww-perl-5.35.

Thanks,


follow-up re handling redirects with cookies and other stuff

Maurice Aubrey (maurice@hevanet.com)
Thu, 16 Jul 1998 23:26:57 -0600 

     Messages sorted by: [ date ][ thread ][ subject ][ author ] 
     Next message: Charles C. Fu: "follow-up re handling redirects with cookies and other stuff" 
     Previous message: Charles C. Fu: "handling redirects with cookies" 



After looking at the code further, I see that cookies are handled
in internal redirects if a cookie jar is associated with the
user agent object.  Perhaps a note should be added to LWP::UserAgent
saying something along the lines of:

   Redirect responses may include cookies, which will be lost if
   the redirection is handled internally and no cookie jar
   has been associated with the object.

Also, the following code 

$url = url '../../images/interface_images/logo.gif', 'http://www.ahn.com/';
$url = $url->abs;
print "$url\n";

produces:

  http://www.ahn.com/../../images/interface_images/logo.gif
  
Is that the correct behavior?  It seems like it should realize it's
impossible, and produce:

  http://www.ahn.com/images/interface_images/logo.gif

which is apparently what Netscape thinks it is.  Or is Netscape broken?

Finally, I noticed that HTML::LinkExtor doesn't find URLs in 
META tags (such as refresh).  From reading the HTML 4.0 spec,
it looks like there's no official specification for the attribute
syntax (yet?) ... it says it's specified by 'profiles' and
'This specification does not define formats for profiles'.
Anyway, I'm wondering if the development of something like
HTML::Meta would be useful, which could understand the profile
format and correctly parse meta tags?  Is anyone working on this?

Thanks,

Re: handling redirects with cookies

Gisle Aas (gisle@aas.no)
17 Jul 1998 13:01:21 +0200 

     Messages sorted by: [ date ][ thread ][ subject ][ author ] 
     Next message: Tony FRADIN: "Installation problems" 
     Previous message: Charles C. Fu: "follow-up re handling redirects with cookies and other stuff" 
     In reply to: Charles C. Fu: "handling redirects with cookies" 



"Charles C. Fu" <ccwf@bacchus.com> writes:

> Maurice Aubrey writes:
> > The site is returning 302, because it wants you to fetch the same
> > page, but wants you to include the cookie the second time around
> > (which is what terminates the redirects).  LWP doesn't seem to do
> > that.
> 
> > Can we improve this?  I'm wondering if, once the redirect is noticed
> > and the request is cloned, the response object could be scanned for
> > cookie headers, and if present, add a cookie jar to the new request?
> > It would also note that cookies were set, and would agree to follow
> > the redirect even if it is identical (and fail only if max redirects
> > is reached).
> 
> Max redirects should be reduced to 5, and we should fail only when
> reaching that limit.  RFC 2068 (HTTP/1.1) says
> 
>    A user agent SHOULD NOT automatically redirect a request more than
>    5 times, since such redirections usually indicate an infinite loop.

draft-ietf-http-v11-spec-rev-03 says:

|  A client SHOULD detect infinite redirection loops, since such
|  loops generate network traffic for each redirection.
|
|    Note: previous versions of this specification recommended a maximum
|    of five redirections. Content developers should be aware that there
|    may be clients that implement such a fixed limitation.

I would be more happy with a solution that considered the headers in
the requests when determining if there is a loop.

Regards,
Gisle