listing ftp directories thru http proxy using libwww

Alex Lewin (lewin@vgi.com)
Fri, 7 Feb 1997 13:37:45 -0500 (EST) 

Hi all,

I am running behind an HTTP-style HTTP etc. proxy server (CERN httpd). I
don't have an FTP-style FTP proxy server.

I want to generate the list of files in a directory on an FTP server, so
that my program can decide which ones it needs.

The URL might be ftp://ftp.netscape.com/pub/, for instance.

When I ask the proxy server to fetch this URL for me, it returns me HTML,
suitable for display in a browser, but not ideal for looping through. I
have written code for looping through (see below), but it's icky and
brittle.

(Is there a better way?)

Curious to hear any thoughts anyone has about this...

Thanks!
Alex
lewin@vgi.com

****

# (should really be a method on HTTP::Response...)
# assumes $ua exists already.
sub HTTPListFtpDirectory ($) {
   my($url) = @_;
   
   # build a list of files on remote system
   $req = new HTTP::Request('GET', $url);
   $res = $ua->request($req);
   if (!$res->is_success()) {
       my($errcode) = $res->code();
       my($errmsg)  = $res->error_as_HTML();
       $errmsg =~ s/\n/ /g;
       warn("HTTP code $errcode: error listing $url: $errmsg");
       return ();
   }

   # This HTML parsing is a real kludge
   return map {$_ => 1}
   ($res->content() =~ m#HREF=\"[^\"]+\">([\w\.]+)</A>#g);
}