Re: Can LWP/libwww to "streaming" read?
Gisle Aas (gisle@aas.no)
03 Oct 1997 12:11:50 +0200
John Chambers <jmc@nexen.com> writes:
> The application I'm looking at needs to grovel thru a bunch of URLs,
> looking for specific things of interest. In most cases, it can
> determine by some simple heuristics that the file it's looking at
> isn't worth any further examination, and can close the connection,
> saving time and net traffic.
>
> Can LWP support this sort of access? Or does it have to read the
> entire thing in first before returning to the caller?
It is supported. You must use the callback variant of $ua->request()
and then you just die() inside the callback when you have seen enough.
The die message goes into the "X-Died" header, the connection to the
server is closed and $ua->request return.
> If so, are there any code examples that illustrate this? Say, a
> little test that will look for "fubar" in the first 10 lines, and if
> that string isn't there, closes the connection and goes on to the
> next URL in <>.
The lwp-download progam does this when it received the SIGINT signal.
I don't have any other examples at hand right now.
Regards,
Gisle