Re: Problems with Parallel User Agent

• Messages sorted by: [ date ][ thread ][ subject ][ author ]
• Next message: steve.kilbane@ind.alstom.com: "Re: Authenticating large, dynamic posts"
• Previous message: larry@n2h2.com: "Re: ParallelUA stability"
• In reply to: Aaron Aw: "Re: HELP: HTTP proxy authentication"
• Next in thread: Doug Monroe: "Re: HELP: HTTP proxy authentication"

> I wrote simply script (attached to this message):
> * it downloads list of URLs (from http://infoseek.icm.edu.pl/root.html)
> * registers first 300 URLs
> * downloads
>
> But it don't work... It returns 'Broken pipe'. Why ?       

Sorry, I can't reproduce your error. The file you're requesting contains
more than 10.000 URLs, and it seems you're registering them all in your
script below (the "$ile < 300" should maybe be "$file++ < 300"?!).

Since it takes a looooong time for me to download stuff from Poland, I
didn't wait to check all 10.000 of them (just downloading the root.html
took ages!). I tried the first 10, and everything worked fine. Maybe try
downloading them in smaller batches, if everything else fails. Or use some
callbacks to print out more DEBUG messages during runtime in order to find
out if a specific site "breaks" the script (it shouldn't really, but then
who knows).

hth

Marc
-- 
Marc Langheinrich
marclang@cs.washington.edu

• Next message: steve.kilbane@ind.alstom.com: "Re: Authenticating large, dynamic posts"
• Previous message: larry@n2h2.com: "Re: ParallelUA stability"
• In reply to: Aaron Aw: "Re: HELP: HTTP proxy authentication"
• Next in thread: Doug Monroe: "Re: HELP: HTTP proxy authentication"