Patch for two WWW::RobotRules bugs
Liam Quinn (liam@htmlhelp.com)
Fri, 20 Apr 2001 13:18:18 -0400 (EDT)
--8323328-1066306788-987787098=:961
Content-Type: TEXT/PLAIN; charset=US-ASCII
Hi,
Attached is a patch for two WWW::RobotRules bugs:
1. If I use <http://www.htmlhelp.com:80/robots.txt> as the robot_txt_uri,
WWW::RobotRules will not disallow access to
<http://www.htmlhelp.com/award/>, but it will disallow access to
<http://www.htmlhelp.com:80/award/>. My patched version compares the host
and port instead of just the authority, so that both the /award/ URIs are
disallowed.
2. If a robots.txt has
User-agent: WDG_SiteValidator
Disallow: /foo
and my robot uses
User-Agent: WDG_SiteValidator/1.2.5
then /foo is not disallowed. The substring comparison in the is_me method
is looking for "WDG_SiteValidator/1.2.5" within "WDG_SiteValidator"
instead of the other way around.
The patch is against
# $Id: RobotRules.pm,v 1.21 2000/04/07 20:17:54 gisle Exp $
--
Liam Quinn
--8323328-1066306788-987787098=:961
Content-Type: TEXT/PLAIN; charset=US-ASCII; name="RobotRules.pm.diff"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.LNX.4.30.0104201318180.961@localhost.localdomain>
Content-Description:
Content-Disposition: attachment; filename="RobotRules.pm.diff"
LS0tIFJvYm90UnVsZXMucG0ub3JpZwlTYXQgQXByIDIyIDIyOjQzOjM4IDIw
MDANCisrKyBSb2JvdFJ1bGVzLnBtCUZyaSBBcHIgMjAgMTI6Mjg6MjkgMjAw
MQ0KQEAgLTgzLDcgKzgzLDcgQEANCiBzdWIgcGFyc2Ugew0KICAgICBteSgk
c2VsZiwgJHJvYm90X3R4dF91cmksICR0eHQsICRmcmVzaF91bnRpbCkgPSBA
XzsNCiAgICAgJHJvYm90X3R4dF91cmkgPSBVUkktPm5ldygiJHJvYm90X3R4
dF91cmkiKTsNCi0gICAgbXkgJG5ldGxvYyA9ICRyb2JvdF90eHRfdXJpLT5h
dXRob3JpdHk7DQorICAgIG15ICRuZXRsb2MgPSAkcm9ib3RfdHh0X3VyaS0+
aG9zdCAuICI6IiAuICRyb2JvdF90eHRfdXJpLT5wb3J0Ow0KIA0KICAgICAk
c2VsZi0+Y2xlYXJfcnVsZXMoJG5ldGxvYyk7DQogICAgICRzZWxmLT5mcmVz
aF91bnRpbCgkbmV0bG9jLCAkZnJlc2hfdW50aWwgfHwgKHRpbWUgKyAzNjUq
MjQqMzYwMCkpOw0KQEAgLTE3Myw3ICsxNzMsNyBAQA0KIHN1YiBpc19tZSB7
DQogICAgIG15KCRzZWxmLCAkdWEpID0gQF87DQogICAgIG15ICRtZSA9ICRz
ZWxmLT5hZ2VudDsNCi0gICAgcmV0dXJuIGluZGV4KGxjKCR1YSksIGxjKCRt
ZSkpID49IDA7DQorICAgIHJldHVybiBpbmRleChsYygkbWUpLCBsYygkdWEp
KSA+PSAwOw0KIH0NCiANCiA9aXRlbSAkcnVsZXMtPmFsbG93ZWQoJHVyaSkN
CkBAIC0xODUsNyArMTg1LDcgQEANCiBzdWIgYWxsb3dlZCB7DQogICAgIG15
KCRzZWxmLCAkdXJpKSA9IEBfOw0KICAgICAkdXJpID0gVVJJLT5uZXcoIiR1
cmkiKTsNCi0gICAgbXkgJG5ldGxvYyA9ICR1cmktPmF1dGhvcml0eTsNCisg
ICAgbXkgJG5ldGxvYyA9ICR1cmktPmhvc3QgLiAiOiIgLiAkdXJpLT5wb3J0
Ow0KIA0KICAgICBteSAkZnJlc2hfdW50aWwgPSAkc2VsZi0+ZnJlc2hfdW50
aWwoJG5ldGxvYyk7DQogICAgIHJldHVybiAtMSBpZiAhZGVmaW5lZCgkZnJl
c2hfdW50aWwpIHx8ICRmcmVzaF91bnRpbCA8IHRpbWU7DQo=
--8323328-1066306788-987787098=:961--