Re: LWP .01

Gisle Aas (aas@oslonett.no)
Tue, 18 Jul 1995 10:25:59 +0200 

> You blew my evening

I hope you did not have much better things to do :-)

> I was playing around with the "get" program, and since I live behind
> a firewall, decided to try and make it work with proxies.  Easy enough,
> except it seems to me that proxying is handled at the wrong level.
> Currently, all proxying is done in UserAgent, and really only kicks in
> if simpleRequest() is called.  Since even a simple program like get
> does the LWP::request() call itself, this doesn't really help very much.
> 
> It would seem to be much more useful to move the proxying code into the
> request modules (where you would need to do this once per protocol), or
> even into the Protocol module, so it would be inherited.  This way, 
> proxying would be completely automatic if needed (and the higher up it
> goes in the class tree, the easier it would be to selectively disable).
> 
> Does this make sense, or am I missing something fundamental?

I must admit that I did not check that proxying worked before releasing the 
code.  I don't have a proxy server handy for the time beeing.  Now that I have 
looked at code code I discovered a few typos releated to proxying.  The 
following patch ought to make things work (but I have not tested it):

--- UserAgent.pm        1995/07/17 10:08:03     1.10
+++ UserAgent.pm        1995/07/18 08:05:42
@@ -216,12 +216,12 @@
     if ($self->useEval) {
         # we eval, and turn dies into responses below
         eval {
-            $response = $protocol->request($request, $proturl, 
+            $response = $protocol->request($request, $proxy, 
                                            $arg, $size, $timeout);
         };
     } else {
         # user has to handle any dies, usually timeouts
-        $response = $protocol->request($request, $proturl,
+        $response = $protocol->request($request, $proxy,
                                        $arg, $size, $timeout);
     }
     alarm(0) if ($self->useAlarm); # no more timeout


The idea is that the UserAgent handles proxying and send the proxy URL in the 
$protocol->request().  There should not be any proxy parameter to the user 
visible $ua->request().  Proxying is then handled by the individual 
Protocol/*.pm module.

The proxy code has moved from place to place during the development.  I think 
the current location is reasonable.  Martijn might be able to elaborate on 
this.

Another matter is that the "get" script ought to have some way to set up 
proxying.  Code donations are appreciated.


> While I'm on the subject of Request, Response and Message, these classes
> seem to be strictly HTTP.  I'm wondering if using these names runs the
> risk of polluting the namespace too fast - maybe they should be prefixed
> with HTTP_, or some version thereof.  This would also make it much easier
> to handle (and test) future versions of HTTP, and might reduce confusion when
> ftp, gopher and whatever make it in.

The basic idea of the libary is to model all protocols as HTTP.  The user 
agent should be able to treat all communications the same way.  The HTTP way 
of doing things ought to be powerfull enough to represent them all.  Isn't 
this the spirit of WWW?

> Also, at first glance there is a minor bug in the get script - I haven't
> really double checked this, but it looks like $ua->timeout($timeout) is called
> before $ua is instantiated, while you are still in the getopts 
> processing.  

Oh yeah,  I moved the code to much around before the release.  Please apply the following patch:

--- get 1995/07/17 09:59:58     1.5
+++ get 1995/07/18 07:33:57     1.6
@@ -1,6 +1,6 @@
 #!/usr/local/bin/perl -w
 
-# $Id: get,v 1.5 1995/07/17 09:59:58 aas Exp $
+# $Id: get,v 1.6 1995/07/18 07:33:57 aas Exp $
 #
 # Simple user agent using LWP library.  It's interface is based on the
 # libwww-perl-0.40 program with the same name.
@@ -115,7 +115,7 @@
 
 
 $0 =~ s,.*/,,;  # use basename only
-$VERSION = sprintf("%d.%02d", q$Revision: 1.5 $ =~ /(\d+)\.(\d+)/);
+$VERSION = sprintf("%d.%02d", q$Revision: 1.6 $ =~ /(\d+)\.(\d+)/);
 
 
 require LWP;
@@ -185,6 +185,9 @@
 usage() if $opt_h || !@ARGV;
 
 
+# Create the user agent object
+$ua = new LWP::UserAgent;
+
 $method = uc($opt_m || $0 || "GET");
 if ($opt_f) {
     if ($opt_c) {
@@ -244,10 +247,6 @@
     die "$0: Can't set Content-type for $method requests\n"
         if defined $opt_c;
 }
-
-
-# Create the user agent object
-$ua = new LWP::UserAgent;
 
 # Set up a request.  We use the same request for all URLs.
 $request = new LWP::Request $method;

Regards,
Gisle