Redirects and Cookies?

Adrian Blakey (ajb@alameda.net)
Sat, 02 Oct 1999 10:15:02 -0700 

I am using WWW::Robot. Several IIS sites are immune (although a browser
with cookies enabled, works fine) and behave in a similar manner, (see
trace below), can someone suggest a workaround? To summarize:

1. Send GET /
2. Returns redirect, and no cookies in the header
3. Submit new request with no cookies
4. Returns redirect.
5. Submit request from the redirect, get nice page that instructs
cookies.

Where are the cookies coming back? why am I not seeing them ?  
 
LWP::RobotUA::simple_request: ()
LWP::UserAgent::simple_request: GET http://www.xxxx.com/
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::http::request: GET / HTTP/1.0
From: root
Host: www.xxxx.com
User-Agent: Getter/0.005

LWP::Protocol::http::request: reading response
LWP::Protocol::http::request: HTTP/1.1 302 Moved Temporarily
Server: Microsoft-IIS/4.0
Date: Sat, 02 Oct 1999 16:47:57 GMT
Connection: close
Location: /?wfx2=lh2hj34l8s

LWP::Protocol::http::request: HTTP/1.1 302 Moved Temporarily
RobotUA back from SUPER::simple_request
RobotUA back from visit
LWP::UserAgent::request: Simple result: Found
LWP::UserAgent::request: ()
LWP::RobotUA::simple_request: ()
LWP::RobotUA::simple_request: Must wait 6 seconds
LWP::UserAgent::simple_request: GET http://www.xxxx.com/?wfx2=lh2hj34l8s
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::http::request: GET /?wfx2=lh2hj34l8s HTTP/1.0
From: root
Host: www.xxxx.com
User-Agent: Getter/0.005

LWP::Protocol::http::request: reading response
LWP::Protocol::http::request: HTTP/1.1 302 Moved Temporarily
Server: Microsoft-IIS/4.0
Date: Sat, 02 Oct 1999 16:48:03 GMT
Connection: close
Location: /_mem_bin/nocookie.asp?/?wfx2=lh2hj34l8s

LWP::Protocol::http::request: HTTP/1.1 302 Moved Temporarily
RobotUA back from SUPER::simple_request
RobotUA back from visit
LWP::UserAgent::request: Simple result: Found
LWP::UserAgent::request: ()
LWP::RobotUA::simple_request: ()
LWP::RobotUA::simple_request: Must wait 6 seconds
LWP::UserAgent::simple_request: GET
http://www.xxxx.com/_mem_bin/nocookie.asp?/?wfx2=lh2hj34l8s
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::http::request: GET
/_mem_bin/nocookie.asp?/?wfx2=lh2hj34l8s HTTP/1.0
From: root
Host: www.xxxx.com
User-Agent: Getter/0.005

LWP::Protocol::http::request: reading response
LWP::Protocol::http::request: HTTP/1.1 200 OK
Server: Microsoft-IIS/4.0
Date: Sat, 02 Oct 1999 16:48:09 GMT
Content-Type: text/html
Cache-control: private

LWP::Protocol::http::request: HTTP/1.1 200 OK
LWP::Protocol::collect: read 1423 bytes
RobotUA back from SUPER::simple_request
RobotUA back from visit
LWP::UserAgent::request: Simple result: OK

Process after GET
http://www.xxxx.com/
    error code 200
Parse http://www.xxxx.com/_mem_bin/nocookie.asp?/?wfx2=lh2hj34l8s into
HTML::TreeBuilder ...
Check META NAME=ROBOTS ...