RE: 500 Errors That Work
Mike Sheldrake (kelly@anet.net)
Fri, 23 Jul 1999 18:06:38 -0700
RE: FTP 500 Errors That Work
When LWP makes FTP requests, it treats the portion of the URL after the last
"/" as a filename and uses the FTP command 'RETR'. If the last portion of
the URL is in fact a directory name, the request will fail, returning a 500
series error. LWP uses this failure to determine if it should make a
directory request using 'NLST' (which it should have done in the first
place). These links that point to directories on FTP servers work fine in a
browser - you get a directory listing like you would expect.
As long as the FTP server returns returns "550" in the situation described
above, LWP will go on and make the 'NLST' request and you'll get your
directory listing. BUT there is at least one FTP server, that is growing in
popularity right now, that returns "553" in this situation. LWP does not
check for this accommodate this error code and will return the error.
The FTP server in question is ProFTPD (http://www.proftpd.org/).
The LWP package involved here is LWP::Protocol::ftp.
The problem seems to be ProFTPD's innappropriate use of error code "553".
I've raised this issue with them and with this list. (No responses yet.) My
original two postings are included below.
- Mike Sheldrake
kelly@anet.net
I looked up RFC 640 (Revised FTP Reply Codes), thanks to Mr. Erskine, and
that eventually lead me to it's descendent, RFC 959 (FILE TRANSFER
PROTOCOL), which seems to be the most recent specification (October 1985).
In RFC 959 I read that the FTP reply code 553 may be returned for the
following commands:
STOR
STOU
APPE
RNTO
The theme seems to be that these are commands that create or change the
state of a file on the FTP server. As Mr. Erskine notes, 553 is not listed
as a possible reply code for the RETR command.
I hope we can get some feedback from the ProFTPD people. It does look like
they should have their server return 550 rather than 553 when a client
attempts a RETR on a directory name.
- Mike Sheldrake
kelly@anet.net
> -----Original Message-----
> From: thomas.erskine-dated-ed553265ce53295d@crc.ca
> [mailto:thomas.erskine-dated-ed553265ce53295d@crc.ca]
> Sent: Wednesday, July 07, 1999 05:20 AM
> To: Mike Sheldrake
> Subject: Re: ProFTPD Reply Code for RETR on a Directory Name
>
>
> On Tue, 6 Jul 1999, Mike Sheldrake wrote:
>
> > I have a PERL FTP script that stopped working, and this is why:
> >
> > When the PERL's LWP::Protocol::ftp tries to GET this URL:
> >
> > ftp://username:password@ftp.myhost.com/mydirectory
> >
> > it first tries to retrieve (RETR) "mydirectory" as if it were a file.
> >
> > When this fails, LWP::Protocol::ftp makes a directory listing
> request (NLST)
> > on "mydirectory", but only if the FTP reply code to the RETR
> equals "550".
> > If it equals anything else, LWP::Protocol::ftp will return a
> "Bad Request"
> > error because the FTP server returns "mydirectory: Not a regular file".
> >
> > My FTP server is replying with "553". LWP::Protocol::ftp does
> not check for
> > this code. Perhaps it should. Or perhaps the FTP server should
> send "550"
> > instead, as others do.
>
> The only relevant RFC I could find (640) lists (among other things) the
> return codes to be returned by the RETR command. 550 and 530 are there,
> but 553 isn't. Unless they can find a more recent supporting RFC, I'd say
> that ProFTPD is out of line here.
>
> > The FTP server software in question: ProFTPD 1.1.7p13
> > The version of LWP::Protocol::ftp in question: 1.24
> >
> > RFC 959, Section 4.2 defines the FTP reply codes like this:
> > 550 Requested action not taken. File unavailable (e.g., file
> not found, no
> > access).
> > 553 Requested action not taken. File name not allowed.
> >
> > The ProFTPD folks decided to use 553, and I don't blame them,
> but "File name
> > not allowed" is ambiguous. Was 553 intended to mean "The filename you
> > requested is not allowed (because there is a problem with the
> filename)." or
> > "You aren't allowed to send a filename with this request."?
> >
> > I don't know who should change their code. I hope to hear of a
> discussion
> > and agreement. In the meantime, I'll get by with my modified version of
> > LWP::Protocol::ftp. Line 179 now reads:
> >
> > } elsif (!length($remote_file) || $ftp->code == 550 ||
> $ftp->code == 553) {
> >
> >
> >
> > - Mike Sheldrake
> > kelly@anet.net
> >
> >
> >
> >
>
> --
> "Life is much too important to be taken seriously."
> Thomas Erskine <thomas.erskine-dated-sig@crc.ca>
> (613) 998-2836
>
>
> -----Original Message-----
> From: libwww-perl-request@ics.uci.edu
> [mailto:libwww-perl-request@ics.uci.edu]On Behalf Of Charles Michael
> Sent: Friday, July 23, 1999 02:43 PM
> To: libwww-perl@ics.uci.edu
> Subject: 500 Errors That Work
>
>
> Hi,
>
> I'm using LWP::UserAgent to test a series of ftp/http links with a
> combination of LWP::UserAgent (to pull the list on links) with
> LWP::Parallel::UserAgent (to do HEAD to check if the links work or not).
>
> Ex: LWP::UserAgent pulls in the html file and grabs all the links found,
> then passes it to a new object, (Parallel UA) to do the head requests
>
> Now, I've encountered 2 critical problems:
> 1- Occassionally (mostly FTP sites), I get a 500 response code
> message, but
> the actual link works fine (very bizarre!).
> 2- Long response requests will not timeout. I know this has been a problem
> for many users, but the problem originates with LWP and not with the
> Parallel UA. The author of PUA has been very helpful although none of the
> patches or suggestions have worked to date. Has anyone been able to fix
> either of these problems?
>
> Thanks everyone.
> PS: Marc, you've been a great help :)
>
> Charles
>
> Charles Michael
> Web Automation Co-op
> GO-DSP, A Texas Instruments Company
> tel. 416-599-6868 ext. 240
>
>