UserAgent with ASP

Michael R. Fahey (mfahey@qilinlaw.com)
Wed, 14 Feb 2001 16:03:17 +0800 

The server returns an error when I request the following url using
LWP::UserAgent.


$url = 'http://someagency.gov.tw/reg/asp/WB0104I_1.asp?ScrollAction=Page 2';
$request = new HTTP::Request('GET', $url);

I can retrieve the first page with:

	$SEARCH_URL = 'http://someagency.gov.tw/reg/asp/WB0104I_1.asp';

	use LWP::UserAgent;
	use HTTP::Cookies;

	use CGI;
	$cgi = new CGI;
	$ua = new LWP::UserAgent;


	my $string_head = shift (@ARGV);
	my $file_name = "$string_head.html";

	open (OUT, "> $file_name") or die "Couldn't open $file_name for reading:
$!\n";

	$request = new HTTP::Request('POST', $SEARCH_URL);
	$request->content_type('application/x-www-form-urlencoded');
	$request->content("english_name=$string_head&basic_select=3&query=¬d¸ß");
#garbage characters here are Chinese (big-5) for 'query'
	$response = $ua->request( $request );


Then I want to get the next five pages, so I tried ...

	$url = 'http://someagency.gov.tw/reg/asp/WB0104I_1.asp?ScrollAction=Page
2';
	$request = new HTTP::Request('GET', $url);      $request->content();
	$response = $ua->request( $request );
	$content=$response->content();
	my $file_name = "$string_head2.html";open (OUT, "> $file_name") or die
"Couldn't open 	$file_name for reading: $!\n";
	print OUT $content;
	print "$content\n\n";

But no luck.  Is my url malformed? Does it need to be escaped?  It appears
in the browser as
http://someagency.gov.tw/reg/asp/WB0104I_1.asp?ScrollAction=Page%202 when I
go to the second page manually.

Thanks for any suggestions.

Michael Fahey