Re: LWP

Bowen Dwelle (bowen@hotwired.com)
Tue, 02 Sep 1997 19:23:13 -0700 

I have some simple code that we use for this. It forks X child processes
that each hit a specified URL; you can use it to time throughput and
latency, at least until you run up against the performace of the test
client machine. 

-------------------------------------snip------------------------------------
#!/usr/local/bin/perl

$usage = <<END;
usage: $0

  --children|c    <int>(1)                 number of concurrent forked
processes to spawn
  --queries|q     <int>(20)                number of queries to run

END

use Benchmark;
use Carp;
use Data::Dumper;
use Getopt::Long;
use FileHandle;
use LWP::UserAgent;
use URI::URL;

$logfile = './load.log';
$LOG = FileHandle->new( ">> $logfile" ) || die "couldn't open logfile:
$logfile\n";

$| = 1;

# arguments

%opt = ();
$result = &GetOptions (
  \%opt,
  "config",
  "debug",
  "help",
  "children|c:i",
  "queries|q:i",
  "host|h:s",
  ) ;

if ( ! $result || $opt{'help'}) {
  print $usage;
  croak "Invalid command line option(s)";
}

$children = $opt{children} || 1;
$queries = $opt{queries} || 20;
$engine = $opt{host} || '';
$use_wordlist = 0;
$root_url = "http://$host";

$queriestotal = $children * $queries;

print <<END;
This is the load test client.

child processes             $children
queries                     $queries
total queries               $queriestotal
root_url                    $root_url
END

croak "--config was used\n" if $opt{config};

#######################################################
#######################################################
#######################################################

# create a bunch of children listening to our semaphore

$time0 = Benchmark->new();

my $i;
for $i (1..$children)
{
  FORK: {
    my $pid;
    unless ($pid = fork)
    {
      if (defined $pid)
      {

        my $countthing = 0;
        my $ua = new LWP::UserAgent;
        print "IN CHILD $i -- $$\n";
        print $PID "$pid\n";                      # record process ID

        srand( time() ^ ($$ + ($$ << 15)));

        for (1..$queries)
        {
          $countthing++;

          my $url = new URI::URL $root_url;
          my $url_string = $url->as_string;
          print "CHILD $i -- query $countthing: $query -- url: $url_string\n";

          my $request = new HTTP::Request('GET',$url);
          my $response = $ua->request($request);
          my $results = $response->title;

##          my $content = $response->content;
##          print "CONTENT:\n$content\n";

          print "CHILD $i -- results: $results\n";
        }

        print "CHILD $i EXITING\n";
        exit 1;

      }
      elsif ($! =~ /No more process/)
      {
        # if we wait we can fork?
        sleep 5; redo FORK;
      }
      else
      {
        # irrecoverable error
        croak "Can't fork\n";
      }
    }
  }
}

#######################################################

for $i (1..$children) { wait; }

$| = 1;

$time1 = Benchmark->new();
print "STOPPING TIMER\n";
my $T = timestr(timediff($time1, $time0));

$PID->close;

$c = $children;
$q = $queries;
$Q = $queriestotal;

eval { $QT = sprintf "%.2f", ( $Q / $T ); };
eval { $TQ = sprintf "%.2f", ( $T / $Q ); };
eval { $Tq = sprintf "%.2f", ( $T / $q ); };

# format $T for human consumption
$hT = 0.001;
$hT = int($1)  if ($T =~ /(.*)secs/);
$hT = 1        if ($hT < 1);

print <<END;

RESULTS

Children (c)	 		$c
Queries/child (q)		$q
Total Queries (Q)		$Q
Total time (T)			$hT
throughput (Q/T)		$QT
time per query (T/Q)		$TQ
user time per query (T/q)	$Tq

END

$D = scalar(localtime);
$logstuff = join ( ',', $c, $Tq, $q, $Q, $hT, $QT, $TQ, $Tq, $D, $host );
print $LOG $logstuff, "\n";

exit;
-------------------------------------snip------------------------------------


At 04:30 PM 9/2/97 -0400, Ledbetter, Bruce wrote:
>
>
>
>I need an application to test the performance of a web server.  We are 
>particularly
>interested in hitting the server with many simultaneous request.   When the 
>site in
>question goes live, we expect it to be hit very hard, and we want to be sure
>it can handle the load.  We would like to try to simulate this before we 
>launch the site.
>
>I've been reading the "Web Client Programming" book, by O'Reilly and it
>seems that I might be able to do what I want in Perl using LWP.   Would 
>anyone
>care to comment on my approach?   Are there other tools out there I should
>know about that do this already?
>
>Thanks,
>Bruce Ledbetter
>Ammirati Puris Lintas
>
>
>

Bowen Dwelle -- 415 276.8607