URI and spidering unique docs

Bill Moseley (moseley@hank.org)
Wed, 28 Mar 2001 23:58:45 -0800


Oh my, I'm writing yet another spider, for some reason.

I'd like to only spider documents one time.  So I'm using a hash of
URI->canonical keys.

Although I realize these *could* be two different docs, they are not on our
server:
     http://localhost/path/to/my/file.html
     http://localhost/path/to/../to/my/file.html

Any (URI?) tricks to seeing those as the same document?


Bill Moseley
mailto:moseley@hank.org