RE: Questions with HTTP Get and LWP

BAZLEY Sebastian (Sebastian.BAZLEY@sema.co.uk)
Tue, 18 May 1999 10:33:02 +0100
Messages sorted by: [ date ][ thread ][ subject ][ author ]
Next message: Lukasz Keler: "Problems with Parallel User Agent"
Previous message: qliang: "Questions with HTTP Get and LWP"
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_000_01BEA115.4FB20BEA
Content-Type: text/plain;
	charset="iso-8859-1"

Some suggestions - hope they work (I have not tried them, and I have not
used Jef's program).

> -----Original Message-----
> From: qliang [mailto:qliang@sims.berkeley.edu]
> Sent: 18 May 1999 00:46
> To: libwww-perl@ics.uci.edu
> Subject: Questions with HTTP Get and LWP
> 
> 
> 
>  Hi,
> 
> I have two questions here:
> 
> 1. I used Jef Pozkanzer's "http_get"
>    "C-language" program program to
>    request HTML pages, and it works fine
>    with most pages, except for a few:
> 
>    e.g., when I used it to get the page
>    "http://www.microsoft.com"
> 
>    I got the following error message:
> 
>    <html><HEAD><BASE HREF=http://www.microsoft.com/><title>Directory
> Listing Denied</title></h
> ead>
>    <body><h1>Directory Listing Denied</h1>This Virtual Directory does
> not allow contents to be
>  listed.</body></html>

You could try using 

	http://www.microsoft.com/

instead - perhaps it does not like the missing trailing slash ?

> 
>    It doesn't know how to deal with
>    some of the Yahoo pages either:
> 
>    when I used it to get the page
>    "http://www.yahoo.com/r/ar"
> 
>    it returned nothing back.
> 

Could this be due to a redirect or refresh ?

When I tried the above URL in a browser, I was taken somewhere different.

>    Interestingly enough, I don't have
>    any problem fetching those pages
>    with LWP. But I don't know why. Could
>    anyone kindly spend some time explain-
>    ing this to me?
> 
> 2. When I used LWP to fetch html pages,
>    it seems that somehow it cannot fetch
>    <...Redirect...> strings properly. Can
>    anyone shed some light on this problem?
> 

Perhaps you can post some more detail on this ?

Redirect is disabled by default for POSTs, but you can override
redirect_ok() in LWP::UserAgent if necessary. Here is my version:

#	Over-ride the redirect_ok method so that POSTs follow redirects:
package LWP::UserAgent;
sub redirect_ok{
    my($self, $request) = @_;
    $request->method("GET") if $request->method eq "POST";
    1;
}


>    Many many thanks for your help.

If not, please ignore my ramblings...

> 
>    Best,
> 
>    Qun Liang
> 
> 


___________________________________________________________________________
This email is confidential and intended solely for the use of the 
individual to whom it is addressed. Any views or opinions presented are 
solely those of the author and do not necessarily represent those of 
Sema Group. 
If you are not the intended recipient, be advised that you have received this
email in error and that any use, dissemination, forwarding, printing, or 
copying of this email is strictly prohibited.

If you have received this email in error please notify the Sema Group
Helpdesk by telephone on +44 (0) 121 627 5600.
___________________________________________________________________________

------_=_NextPart_000_01BEA115.4FB20BEA
Content-Type: application/ms-tnef
Content-Transfer-Encoding: base64

eJ8+IjYKAQaQCAAEAAAAAAABAAEAAQeQBgAIAAAA5AQAAAAAAADoAAEIgAcAGAAAAElQTS5NaWNy
b3NvZnQgTWFpbC5Ob3RlADEIAQSAAQAkAAAAUkU6IFF1ZXN0aW9ucyB3aXRoIEhUVFAgR2V0IGFu
ZCBMV1AAngsBCYABACEAAABFMzI2QzU3OTAwMEREMzExOEVGMDAwQzA0RjdENTUwRgAQBwEggAMA
DgAAAM8HBQASAAsAAAAzAAIALQEBBYADAA4AAADPBwUAEgAKACEAAgACABwBAQ2ABAACAAAAAgAC
AAEDkAYAlAsAADAAAAALAAIAAQAAAAsAKwAAAAAAAwAuAAAAAABAADkAQCPvbBGhvgEeAHAAAQAA
ACAAAABRdWVzdGlvbnMgd2l0aCBIVFRQIEdldCBhbmQgTFdQAAIBcQABAAAAGwAAAAG+oMuzOzVT
uHAMvhHTsWUAoMnOAOUAEPelQAACAQkQAQAAAFMGAABPBgAADwsAAExaRnUOIRgkAwAKAHJjcGcx
MjXiMgNDdGV4BUEBAwH3/wqAAqQD5AcTAoAP8wBQBFY/CFUHshElDlEDAQIAY2jhCsBzZXQyBgAG
wxEl9jMERhO3MBIsETMI7wn3tjsYHw4wNREiDGBjAFCzCwkBZDM2FlALp2MBMAMGAANwZSBzdWdn
GQeQdGkCIAQgLSBoBG9wHWB0aGV5IEJ3BbBrIChJHlBhUnYdYG5vBUB0CIFkcR6ibSwgAHAgUB9p
dasUECBQSgEQJwQgcANg4QnAYW0pLgqiCoQKgNY+HjAj4k8FEGcLgAdAcwXQB5BzYR2wI+MjZkYh
A2E6IHFsBzBuZ4wgWwDAAxB0bzomZEJAAJBtcy5iBJBrlGVsHtAuCYB1XSNmpwZgAjAmQDE4BdBh
HuAEMTkqQCAwMDo0KjYjZlQnMCAmcGJ3xSvALR6AcmxADeAn8I0a0GkokijndWJqBZA9KZFRClAd
1QPwHrAgSGBUVFAgRxQgIMNMPFdQI2YwDyN1LvBpLPswjh9ldB8AJlAuJx7AGCAyOjCOMS4hASHG
IFBob3prAHB6BJAiQSKSaAJAcF8dsHQiMYexMfAiQy0LYCagdSUB/iIiZjmXJyA4KRggM/Mu8exN
TCJgJQFzILQuwB7z9QQgZguAZTgpLrMEYDvBczxFDsBjZQUxAhAFwGEPPXAH0DTfOJFlLmcufSCw
dx7AA6A2FTzxJyAgHzfhHqI8MzgqN5I6Ly+ZK8EubQ3gA2BzbwGAui4FoG04FzgpH2BnH+L7Q5EC
EGwXsAPwJqEEkANgXwXAB4Ek8kCPOJE8N5BtgGw+PEhFQURLIBBCQVNFLvBSRUa2PUTPRdUvSyAd
4HQoYLg+RGkYIC3QBbB5I2ZuTAQAHeAmoUQJ8CAxPBYvTfRQQGgjZmVhZCY+SioG4GR5SyBoMSdO
SE9fUtFUaAQAIFadTmB0OUADIFMIZG8HkO8jZh/SB0BIUSAFoAIwKXF/BCBDESgQMYcmcB3QCYAu
31BAUnRQ0EryIwpZCGBXkfx1bCBRU3EhwEiCIwoMgv8eUExfTWUjCguAWTFRgB4x+ywBE+BwBCA8
8VZCH8MmcP8oQENzRXAEEEiCIBAm8UiC6nMLYHMu4D8jDEbaYPT0bicFQGsf0AfgHmAH4P9DEQEA
JJEusjgpRcAdUUXQ+UNzWWEeYDPQPEM/cC7BPwSQSZ8+QkI/Q09EXy5582jCTYNyLwrARk84kTzx
fxggVVAEoCBBH9FU4CahYv0A0Gsi9TDmIwQIUVuSVOL/KBBWMApQa/JAMBghTmNoMN8FwBggA1AH
kGONV2skIBfnIMAG4B+hVVI8IAuAQCFqYgNgdxQQciCwH2B3+2NgHqBhKEADoGfyaxEYIP9WMAaQ
QFAYIAIwIv1HUlfBO3WxT5JsHuAJ8AhgZ2j/eUJWQGWyH4I4KQBwHuAicfcCYCCQQEF0E9Bicx5g
FBDPPDQ9zi/RNfBCdQVAfdb/ZfNrECiAEiJbkH6cAiAdYOprC4BkfSFzHoAg4Wfz/x3gHVEOwAtT
JXdwUoBjVPHvbAEHgGOlMOgyNfB21SHD/y/Ra/KAA11hSwA8NW/cFBB/IJBYERPgjSEdQWZCPPFj
rwBwH9KAA0oqLpAgUnTV/5AhI8Ad0AUQJqAiUywCg8L/AHCETx1wHsCGFSZwfZB1Md8DoHOzf4WI
7SMEUGB1hRD7W0EDkXA+0mfzBGB6khQg/ybxlIdjq5BWPOAEIHTgJPDbf7EgUGJWIQEQYVuAP9TQ
UE9TVDyBYoKhl1YbeAFI0GkBAHS3X29rjCgpeHIv0To6VXkR/kEdsAIwPOA2oD2gP6Ak4dtOwDXw
SHqCVPFtHuCeAa8AkAIgadUjBCNdI0+eAf4tnjNscp6JSRGAkYYSjZT3nMNIFXS3c2nVCrBx4CUB
v5+NGTAjEx2AXVCeiVwAACcjEziSoeAoJBQQbGYNILAkO3WfQD0gQF8nqXU4kqw2LT6lVCgi8EdF
VCKfQTagrc4/cFZxN3CcwiKtGTGpdVz+fVxrODgp8H9RA4Ee4I2hPm49UgWxl1EFwB7AbHD9IvtJ
oJEf4CCwC1BRcIDB/yRQH9B6kaHhIrECYJFSkCH7Y784VkIdwTI/OJEuEJ9xFyaCMI8jE3294AAD
AP0/UgMAAB4AQhABAAAAJgAAADwzNzQwQUEyMC5BNDQxRDk1NEBzaW1zLmJlcmtlbGV5LmVkdT4A
AAADAN4/r28AAAsAAIAIIAYAAAAAAMAAAAAAAABGAAAAAAOFAAAAAAAAAwAIgAggBgAAAAAAwAAA
AAAAAEYAAAAAUoUAAPATAAAeAAmACCAGAAAAAADAAAAAAAAARgAAAABUhQAAAQAAAAQAAAA4LjUA
CwAKgAggBgAAAAAAwAAAAAAAAEYAAAAABoUAAAAAAAADAAuACCAGAAAAAADAAAAAAAAARgAAAAAB
hQAAAAAAAAsAFIAIIAYAAAAAAMAAAAAAAABGAAAAAA6FAAAAAAAAAwACgAggBgAAAAAAwAAAAAAA
AEYAAAAAEIUAAAAAAAADABWACCAGAAAAAADAAAAAAAAARgAAAAARhQAAAAAAAAMAF4AIIAYAAAAA
AMAAAAAAAABGAAAAABiFAAAAAAAAHgAmgAggBgAAAAAAwAAAAAAAAEYAAAAANoUAAAEAAAABAAAA
AAAAAB4AJ4AIIAYAAAAAAMAAAAAAAABGAAAAADeFAAABAAAAAQAAAAAAAAAeACiACCAGAAAAAADA
AAAAAAAARgAAAAA4hQAAAQAAAAEAAAAAAAAACwAzgAsgBgAAAAAAwAAAAAAAAEYAAAAAAIgAAAAA
AAALADWACyAGAAAAAADAAAAAAAAARgAAAAAFiAAAAAAAAAMAJgAAAAAAAwA2AAAAAAAeADFAAQAA
AA0AAAA4ODNTNzFENUVEMTkAAAAAAwAaQAAAAAAeADBAAQAAAA0AAAA4ODNTNzFENUVEMTkAAAAA
AwAZQAAAAAADAIAQ/////wIB+T8BAAAAUwAAAAAAAADcp0DIwEIQGrS5CAArL+GCAQAAAAYAAAAv
Tz1TRU1BL09VPVVLTE9OL0NOPVVLTE9OIE1TTUFJTCBVU0VSUy9DTj04ODNTNzFENUVEMTkAAB4A
+D8BAAAAEQAAAEJBWkxFWSBTZWJhc3RpYW4AAAAAHgA4QAEAAAANAAAAODgzUzcxRDVFRDE5AAAA
AAIB+z8BAAAAUwAAAAAAAADcp0DIwEIQGrS5CAArL+GCAQAAAAYAAAAvTz1TRU1BL09VPVVLTE9O
L0NOPVVLTE9OIE1TTUFJTCBVU0VSUy9DTj04ODNTNzFENUVEMTkAAB4A+j8BAAAAEQAAAEJBWkxF
WSBTZWJhc3RpYW4AAAAAHgA5QAEAAAANAAAAODgzUzcxRDVFRDE5AAAAAEAABzBA1s2RD6G+AUAA
CDDqC7JPFaG+AR4APQABAAAABQAAAFJFOiAAAAAAHgAdDgEAAAAgAAAAUXVlc3Rpb25zIHdpdGgg
SFRUUCBHZXQgYW5kIExXUAALACkAAAAAAAsAIwAAAAAAAwAGEDDV/3MDAAcQCQYAAAMAEBAAAAAA
AwAREAIAAAAeAAgQAQAAAGUAAABTT01FU1VHR0VTVElPTlMtSE9QRVRIRVlXT1JLKElIQVZFTk9U
VFJJRURUSEVNLEFORElIQVZFTk9UVVNFREpFRlNQUk9HUkFNKS0tLS0tT1JJR0lOQUxNRVNTQUdF
LS0tLS1GAAAAAIAL

------_=_NextPart_000_01BEA115.4FB20BEA--
Next message: Lukasz Keler: "Problems with Parallel User Agent"
Previous message: qliang: "Questions with HTTP Get and LWP"