Re: LWP .01
Martijn Koster (m.koster@nexor.co.uk)
Tue, 18 Jul 1995 20:05:26 +0100
In message <199507181714.NAA02311@fugit.ny.jpmorgan.com>, Joel Scotkin writes:
> I also took another good look at the code and have cleared up my own
> confusion.
Ah, classic case of crossed mail :-) I'm glad it is clearer now.
> That said, here is a basically five line patch to "get" which adds a
> new option, -p <proxy>, to enable you to use a proxy in requests. I
> hardcoded in http as the method to proxy
'scheme' you mean. Methods are things like 'GET'.
> - I wasn't sure if that line should contain the whole list of
> methods, or should try to grab the method from the URL and work on
> that.
If you have a proxy which does most protocols, then you might want to
specify a proxy for all transfers. I think this may be quite likely.
On the other end of the scale you'd want:
get -p http http://cache/ -p gopher http://cern/ url
Actually I'd quite like to see a cache config file, but that's
another story.
> One other comment - I noticed that requests seem to go out with a header
> like this:
>
> GET http://archive.ny.jpmorgan.com/
> Accept: */*
> User-Agent: LWP get/1.06
That doesn't happen for me:
normal:
LWP::Socket::write: >>>GET / HTTP/1.0
Accept: */*
User-Agent: LWP get/1.06
<<<
proxying:
LWP::Socket::write: >>>GET http://web/ HTTP/1.0
Accept: */*
User-Agent: LWP get/1.06
<<<
I've appended a slightly different patch, which uses the proxy
for all schemes requested, and adds a '-x' to give full debugging
output. This is delta the original distribution.
-- Martijn
__________
Internet: m.koster@nexor.co.uk
X-400: C=GB; A= ; P=Nexor; O=Nexor; S=koster; I=M
WWW: http://web.nexor.co.uk/mak/mak.html
*** ../lwp-0.01.orig/bin/get Mon Jul 17 11:17:37 1995
--- bin/get Tue Jul 18 20:04:02 1995
***************
*** 1,6 ****
#!/usr/local/bin/perl -w
! # $Id: get,v 1.5 1995/07/17 09:59:58 aas Exp $
#
# Simple user agent using LWP library. It's interface is based on the
# libwww-perl-0.40 program with the same name.
--- 1,6 ----
#!/usr/local/bin/perl -w
! # $Id: get,v 1.6 1995/07/18 07:33:57 aas Exp $
#
# Simple user agent using LWP library. It's interface is based on the
# libwww-perl-0.40 program with the same name.
***************
*** 11,18 ****
=head1 SYNOPSIS
! get [-eEdvh] [-m method] [-b <base URL>] [-t <timeout>]
! [-i <if-modified-since>] [-c <content-type>] <url>...
=head1 DESCRIPTION
--- 11,19 ----
=head1 SYNOPSIS
! get [-eEdvhx] [-m method] [-b <base URL>] [-t <timeout>]
! [-i <if-modified-since>] [-c <content-type>]
! [-p <proxy-url>] <url>...
=head1 DESCRIPTION
***************
*** 64,69 ****
--- 65,74 ----
Content-Type for POST is C<application/x-www-form-urlencoded>. The
default Content-type for the others are C<text/plain>.
+ =item -p <proxy-url>
+
+ Set the proxy to be used for the request(s).
+
=back
The following options controls what is displayed by the program:
***************
*** 98,103 ****
--- 103,112 ----
Print usage message and quit.
+ =item -x
+
+ Extra debugging output.
+
=back
Because this program is implemented using the LWP library, it will
***************
*** 115,121 ****
$0 =~ s,.*/,,; # use basename only
! $VERSION = sprintf("%d.%02d", q$Revision: 1.5 $ =~ /(\d+)\.(\d+)/);
require LWP;
--- 124,130 ----
$0 =~ s,.*/,,; # use basename only
! $VERSION = sprintf("%d.%02d", q$Revision: 1.6 $ =~ /(\d+)\.(\d+)/);
require LWP;
***************
*** 122,129 ****
require LWP::Date;
require URI::URL;
! #require LWP::Debug;
! #LWP::Debug::level('+trace');
# This table lists the methods that are allowed. It should really be
--- 131,137 ----
require LWP::Date;
require URI::URL;
! require LWP::Debug;
# This table lists the methods that are allowed. It should really be
***************
*** 166,172 ****
$opt_h = undef; # print usage
$opt_v = undef; # print version
! unless (getopts("hvuUsedb:t:i:c:m:f")) {
usage();
}
--- 174,183 ----
$opt_h = undef; # print usage
$opt_v = undef; # print version
! $opt_x = undef; # extra debugging info
! $opt_p = undef; # proxy URL
!
! unless (getopts("xhvuUsedp:b:t:i:c:m:f")) {
usage();
}
***************
*** 184,189 ****
--- 195,204 ----
usage() if $opt_h || !@ARGV;
+ LWP::Debug::level('+') if $opt_x;
+
+ # Create the user agent object
+ $ua = new LWP::UserAgent;
$method = uc($opt_m || $0 || "GET");
if ($opt_f) {
***************
*** 245,254 ****
if defined $opt_c;
}
-
- # Create the user agent object
- $ua = new LWP::UserAgent;
-
# Set up a request. We use the same request for all URLs.
$request = new LWP::Request $method;
$request->header('If-Modified-Since', $opt_i) if defined $opt_i;
--- 260,265 ----
***************
*** 265,276 ****
# Ok, now we perform the requests, one URL at a time
while ($url = shift) {
- $request->url(new URI::URL $url, $opt_b);
$response = $ua->request($request);
if ($opt_u || $opt_U) {
! my $url = $request->url->as_string;
print "$method $url\n";
print $request->headerAsString, "\n" if $opt_U;
}
--- 276,292 ----
# Ok, now we perform the requests, one URL at a time
while ($url = shift) {
+ $url = new URI::URL $url;
+
+ $ua->proxy($url->scheme, $opt_p) if $opt_p;
+
+ $request->url($url, $opt_b);
+
$response = $ua->request($request);
if ($opt_u || $opt_U) {
! my $url = $url->as_string;
print "$method $url\n";
print $request->headerAsString, "\n" if $opt_U;
}
***************
*** 311,316 ****
--- 327,333 ----
-t <timeout> Set timeout value
-i <time> Set the If-Modified-Since header on the request
-c <conttype> use this content-type for POST, PUT, CHECKIN
+ -p <proxyurl> use this as a proxy
-u Display method and URL before any response
-U Display request headers after method and URL
***************
*** 320,324 ****
--- 337,343 ----
-v Show program version
-h Print this message
+
+ -x Extra debugging output
EOT
}