Performance Tuning of TWiki

NOTE: Also see more recent PerformanceAnalysis.

Today I did some tests with the built in Perl function times and found some bottlenecks. Commited improvements to TWikiAlphaRelease.

TWiki.pm has now a new writeDebugTimes function to time CPU usage. The result is in file debug.txt. To use it add writeDebugTimes( "funcName" ) just after the functions you want to time, i.e.

    $text= &TWiki::Store::readTopic( $topic );
    &TWiki::writeDebugTimes( "readTopic" );
    $text = &TWiki::handleCommonTags( $text, $topic );
    &TWiki::writeDebugTimes( "handleCommonTags" );
    $text = &TWiki::getRenderedVersion( $text );
    &TWiki::writeDebugTimes( "getRenderedVersion" );

Here is a sample output:

=====       sec:  (delta:)         sec:  (delta:)  function:
===== user: 0.240 (0.240), system: 0.060 (0.060),  start
===== user: 0.420 (0.180), system: 0.090 (0.030),  initialize
===== user: 0.420 (0.000), system: 0.100 (0.010),  readWebTopic
===== user: 0.560 (0.140), system: 0.110 (0.010),  handleCommonTags
===== user: 0.700 (0.140), system: 0.110 (0.000),  getRenderedVersion

In case you want to add this to your TWiki installation, here are the changes you need to do.

Changes in the beginning of TWiki.pm:

use vars qw(
        $debugUserTime $debugSystemTime
    );

$debugUserTime   = 0;
$debugSystemTime = 0;

New sub below sub writeDebug in TWiki.pm:

# =========================
sub writeDebugTimes
{
    my( $text ) = @_;

    if( ! $debugUserTime ) {
        writeDebug( "=====       sec:  (delta:)         sec:  (delta:)  function:" );
    }
    my( $puser, $psystem, $cuser, $csystem ) = times();
    my $duser = $puser - $debugUserTime;
    my $dsystem = $psystem - $debugSystemTime;
    my $times = sprintf( "user: %1.3f (%1.3f), system: %1.3f (%1.3f)",
                  $puser, $duser, $psystem, $dsystem );
    $debugUserTime   = $puser;
    $debugSystemTime = $psystem;

    writeDebug( "===== $times,  $text" );
}

Related Topics: PackageTWikiStore, SpeedyCGI, TWikiWithModPerl

-- PeterThoeny - 06 Mar 2001

Here is an overall method using the perl standard module Benchmark. It's not fine grain but useful for testing different approachs to the code. Particularily for the OOTWiki work. It should work on any systems with /dev/null, not sure if Windows has an equivalent.

Additional code in bin/view:

$query= new CGI;

##### for debug only: Remove next 2 comments (but redirect does not work)
#open(STDERR,'>&STDOUT'); # redirect error to browser
#$| = 1;                  # no buffering

use Benchmark;
print "Content-type: text/plain\n\n";
print "Starting test...\n";

open(NULL,'>/dev/null');
my $fh = select NULL;  # ie. dump STDOUT;

my $t = timeit( 100 , \&main );
select $fh;

print "Finished test.  Results are:\n";
print timestr($t);
 #&main();

sub main
{    
[...]

See http://search.cpan.org/search?mode=module&query=benchmark for further information about the standard perl Benchmark module.

-- NicholasLee - 11 May 2001

The equivalent on Windows is the NUL: device, ie.

echo "No one is listening" > NUL:

Also, in case you care, Windows 95 cannot redirect stderr with "2>&1", it does work on Windows NT.

It might be an idea to abstract out all OS dependent stuff into a separate module.

-- MartinCleaver - 13 May 2001

Something like open(STDERR, ">STDOUT") would probably be sufficent in perl for "2>&1". I assume open(NULL,">NUL") works as expected in win32 perl?

Note, the above is purely for development, not really part of the core TWiki code. So there is no need to worry about seperate modules for this. I was planning to put together a generic debug module, but dealing with the getting the caller package right/etc made me put it off for a while.

-- NicholasLee - 13 May 2001