Re: bug in ParallelUA?
Marc Langheinrich (marclang@cs.washington.edu)
Fri, 11 Sep 1998 09:26:25 +0900
>Hi -- I was writing some code using LWP::Parallel::UserAgent ...
> ... with a callback function that looked
>something like this excerpt from col27:
>
> http://www.stonehenge.com/merlyn/WebTechniques/col27.listing.txt
>
>sub callback_for_parse {
> [... set up ...]
>
> if (length $content) {
> $response->add_content($content);
> [... do some stuff ...]
> return length $content; # go get some more
> }
>
> [... do some different stuff ...]
> $response->content(""); # discard it (free up memory)
> return C_ENDCON; # no more data from here
>}
>
>In the above, if length($content) == 0, this will terminate the
>connection. However, in practice this does not appear to be a sufficent
>condition to show the connection is dead.
Paul,
you're not the only one who has run into this problem. After Randal's column
I have received a number of such reports. Here's what's happening: sysread
apparently returns 0 bytes every so often on a socket that's ready for
reading (as indicated by select). I don't know why select sometimes flags
sockets that don't contain any incoming data (maybe some UNIX hackers can
help?). I put a similar "heuristic" into ParallelUA itself, so that it won't
cut the connection even though 0 bytes arrive on the reading socket.
HOWEVER, my documentation is not only poor but confusing enough, prompting
Randal (I believe) to add the "return C_ENDCON" at the end of his function.
Unless you want to save bandwidth and cut a connection before all the data
has arrived, you don't need to return the "C_ENDCON" flag! Just return undef
(i.e. "return") and ParallelUA will continue with this connection until it's
closed for good. Bottom line: instead of trying to detect EOF inside your
callback, just leave it to ParallelUA. Only if you need just the first half
of an otherwise large response you should be using "return C_ENDCON" to
prematurely end a connection.
Marc