Bug in Protocol::http->request()

Steve Goldstein--Genetics Computer Group (steveg@gcg.com)
Fri, 22 May 1998 12:08:29 -0500 

I've written to this list a few times about this problem:  I've
finally figured out the defect and I have a work-around.

The symptom:

        Platform-dependent, intermittent failure of an LWP client when
        making an HTTP request to a server that requires
        authorization.
 
        The symptom sometimes shows up as an unitialized value called
from LWP::Protocol::http->_get_sock_info() and with an error message in
HTTP::Response->as_string:

500 (Internal Server Error) Bad arg length for Socket::unpack_sockaddr_in, lengt
h is 0, should be 16.

Other times the symptom is that the Response->as_string error message is:
500 (Internal Server Error) Connection reset by peer.

I see the failure once in a while when the server (NCSA http server)
platform is Dec Alpha, almost all the time on Sun, and never on SGI.

The cause (I think):

        The http->request() method opens a socket and makes the
request.  In this case, authentication is required so the response is a
401 Unauthorized error.  Now the http->request() method reads this
from the socket, and then it merrily goes on its way, trying to finish
reading from the socket and then trying to close it.  After that, the
UserAgent->request() method is supposed to detect this error, shove
the credentials into the header, and resubmit the request.  However,
it appears that the server is detecting that the client request is
unauthorized and attempts to close the socket.  Sometimes, the server
closes the socket while the http->request() method is still
processing. Hence, an error is generated by a read or a getpeername()
call to the closed socket.

The work-around:

        In my client, I know that authentication is always required, so
I put the authentication into the header right away, copying the code
from LWP::Authen::Basic        


  require MIME::Base64;
  my ($user, $pass) = ( $self->URL()->user(), $self->URL()->password() );
  my $auth_header = "Authorization";
  my $auth_value = "Basic " . MIME::Base64::encode("$user:$pass", "");
  $request->header($auth_header => $auth_value);


=========================================
I'm sure there are more elegant fixes, but I'll leave that to others.

What I don't understand is why doesn't http->request() just slurp up
the whole buffer at once.  Why does it have to keep poking at the
socket?  If it didn't, there would be a simple fix:

In http.pm-> request() replace  
            $self->_get_sock_info($response, $socket);
        and 
            $socket->close;
with 
            $socket->_get_sock_info($response,$socket)  
              unless $self->_closed_already($socket);
        and
            $socket->close unless $self->_closed_already($socket);

and add:

sub _closed_already
{
    my ($self, $sock) = @_;
    if (stat $sock)
    {
        return 0;
    }
    else
    {
        return 1;
    }
}


	Steve Goldstein