Re: splitlog.pl resources exhausted
Roy T. Fielding (fielding@kiwi.ICS.UCI.EDU)
Sun, 02 Mar 1997 22:08:49 -0800
>splitlog is exiting with the following error message...
>
>Allocation too large at ./splitlog.pl line 714.
>
> while (($ipnum, $value) = each %DNScache)
> {
> ($host, $seen) = split(/\|/, $value);
> if ($StartTime > ($seen + $DNSexpires)) {
>line 714 --> push(@expired, $ipnum);
> }
> }
>
>Indicating that the dnscache.db file is probably too large at 6Mb.
>-rw------- 1 wwwadmin users 5947392 Mar 2 12:52 dnscache.db
Yep. The problem is that splitlog is trying to expire more entries
than it is capable of handling in one go. The reason why it is trying
to remember what needs to expire, instead of just deleting the entry,
is because perl's each() iterator requires that the array not be changed
during the iteration.
A solution would be to add an arbitrary stop value to the loop, e.g.
@expired = ();
while (($#expired < 10000) && (($ipnum, $value) = each %DNScache))
{
...
would limit it to 10000 expirations per running of splitlog. You might
want to increase or decrease that number, depending on available memory.
The same change would apply for wwwstat. Let us know if it works.
...Roy T. Fielding
Department of Information & Computer Science (fielding@ics.uci.edu)
University of California, Irvine, CA 92697-3425 fax:+1(714)824-4056
http://www.ics.uci.edu/~fielding/