Re: accessing URLs that require uid/password

Tak (tak@stargate.jpl.nasa.gov)
Thu, 26 Oct 95 18:20:05 PDT 

Paul,
    Actually, I got something hacked together with some help from
someone else on the list.  Lemme see if I can find the relevant messages.

Found it!  It worked like a charm too.  Not the most elegant of
solutions, as you basically have to intercept an encrypted password,
then hard code it into the momspider routine.  But it works.


> Mark,
>
> Were you able to figure out how to access URLs that require
> a uid/password through a Perl script?  If so, would you be
> willing to share your code?
> Thanks,
> -Paul


On Fri, 5 May 1995, Tak wrote:

> Chris.
>     WOW. That would be great!  I'd love to hear/see what you've managed
> to put together.   Sned it on over.  If you want to dump source code
> you can email it or use my ftp site;
>     <snip>
> 
>     Thanks alot!
> 
>     	tak
> 

Okay, you asked for it. I'll keep it as simple as possible and probably 
talk under your intelligence. Don't take offense.

The Basic protection scheme is outlined at 
http://www.w3.org/hypertext/WWW/AccessAuthorization/Basic.html
Basic steps are:
1. Client requests document
2. Server sends back unauthorized status (reponse code 401 Unauthorized)
3. Client requests document again, this time sending username and password

The only difference between step 1 and step 3 is the following line is 
added to the request:
Authorization: Basic bWFuZ286

'Basic' means the basic encryption scheme is used, and 'bWFuZ286' is 
name:password in encrypted format.  Other encryption schemes can be used 
(Pubkey, PGP, Kerberos), but Basic is the built-in 'standard'. The line 
of code to implement sending this information in libwww is simple:

$headers{'Authorization'} = 'Basic bWFuZ286';

I added this line right before the &www'request. Unfortunately, I know 
nothing about encryption, so I cannot transform username:password into 
the appropriate sequence, perhaps you can figure it out from the above 
http address (something about RFC 1421 *shrug*). Anyway, since I couldn't 
encrypt the username and password myself, I let Netscape do it for me.
Here's how.

I took the code for TinyHTTPD, a Perl HTTP daemon, from
http://rzstud1.rz.uni-karlsruhe.de/~uknf/sw/tinyhttpd.html
and modified it so that instead of returning code 403 for not finding a 
file, it returns code 401. Also, I have it dump the entire contents of 
document requests it receives into a file.  Then, I used to Netscape to 
connect to the daemon, trying to get a file that doesn't exist (any one 
will do).  When Netscape gets the response code 401, it thinks it needs 
to send authentication and prompts for username and password.  I entered 
the username and password I wanted, then let Netscape send it to the 
server. Here's the output in my logfile:

GET / HTTP/1.0
User-Agent: Mozilla/1.1N (Macintosh; I; PPC)
Accept: */*
Accept: image/gif
Accept: image/x-xbitmap
Accept: image/jpeg
Authorization: Basic bWFuZ286

So, that explains how I know what encrypted code to send.  Since I only 
needed to know one username and password, this has worked for me, but of 
course isn't the best solution. The encryption sequence is probably out 
there somewhere, just to find it! I tried looking in the Lynx source 
code, but no luck.

Anyway, this has been quite educational. Now I know that
$headers{'Header'} = "value";
will cause a
Header : value
line to be added to the headers of the http request. If you figure out 
that pesky encryption scheme, e-mail me.  Good luck!

Chris Hilton


----- end of included file----

    Tak


------------------------+-----------------------------------------------------+
Mark Takacs 	    	|  JLP: "You're not God!"   	    	    	      |
tak@jpl.nasa.gov	|    Q: "Blashemy! You're lucky I don't cast you out, |
NASA-JPL - Pasadena CA 	|   	 or smite you or something!"  	    	      |
http://stargate.jpl.nasa.gov/~tak/
------------------------+-----------------------------------------------------+