Re: Out of memory
Ilya Martynov (m_ilya@agava.com)
18 Apr 2001 16:45:06 +0400
MK> Hi!
MK> I have a simple program which opens each URL in a file and saves the result
MK> (actually the HTML code of the requested document) into a data file.
MK> Something like:
MK> my $webdoc = $browser->request(HTTP::Request->new(GET => $url));
MK> ...
MK> and then
MK> my $string = $webdoc->content;
MK> ...
MK> print FILE $string;
MK> ...
MK> But if I try to reach some URL's there an error emerges - out of memory.
MK> I tried to save the content of the following URL's:
MK> http://razor.fri.uni-lj.si:8080/misterP/bin/jpegpush2
MK> http://home.izum.si/cobiss/konference/konf_2000/video/13.wmv
MK> And it doesn't work - there is an out of memory error.
MK> I think that that is because the program tries to download a lot of data -
MK> video stream or big application... My question is: how to avoid this out of
MK> memory error?
Download and write to file by small parts.
Something like
$browser->request(HTTP::Request->new(GET => $url), \&callback, 4096);
sub callback {
my($data, $response, $protocol) = @;
print FILE $data;
}
Or use LWP::UserAgent feature to write directly to file:
$browser->request(HTTP::Request->new(GET => $url, $filename);
See perldoc LWP::UserAgent for details.
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
| Ilya Martynov (http://martynov.org/) |
| GnuPG 1024D/323BDEE6 D7F7 561E 4C1D 8A15 8E80 E4AE BE1A 53EB 323B DEE6 |
| AGAVA Software Company (http://www.agava.com/) |
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-