Question

Hi all,

I'm starting a wiki in a fairly un-wiki-ish way, in that I'm uploading large numbers of documents to use as the seed for TWiki-based knowledgebase. I managed to speed up page views fairly significantly last week by the addition of mod_perl, however I believe this may have caused problems.

After a few hundred consecutive topic creations, the server's host machine slows down significantly, and if left to the extreme, leads to 'can't fork' errors when trying to view some TWiki topics. I'm having to periodically pause the script I have doing the topic uploads, restart httpd, and un-pause the script. If left without this treatment the machine grinds to a sticky halt, so I'm thinking memory leak.

Is this a documented issue? Is there a solution? Should I not be using mod_perl?

Cheers,

Jon

Environment

-- JonathanManning - 26 May 2008

Answer

If you answer a question - or someone answered one of your questions - please remember to edit the page and set the status to answered. The status selector is below the edit box.

Using mod_perl should of course be fully supported. I am using it happily on a virtual machine with 256MB of RAM, and it hardly ever touches its swap area even after weeks of uptime. My installation is based on version 4.0, but I think that some provisions have been made in the code against memory leaks since then.

There are some things you could check:

1. Does your Apache configuration limit the number of requests per thread (say 1000)? That could help to make a small leak harmless, with neglegible effect on performance.
2. Does your WWW::Mechanize application re-send the cookie which TWiki presents? If it doesn't, every request will create a new TWiki session, the unused remains of which need to be harvested later. This has a known (but more or less unavoidable) performance penalty. A huge number of session files in TWiki's working directory could be an indicator for that.
3. Does your WWW::Mechanize application run requests in parallel? If so, many mod_perl processes will be created by Apache, each with a probably big memory footprint. This memory footprint can be considerably reduced by compiling all your modules in your mod_perl startup file.

If those workarounds fail, someone would be in need to check why HowToTrackDownCircularReferences works only for 4.0, and how to adjust it for 4.2

-- HaraldJoerg - 26 May 2008

Those all sound like promising leads- thank you for your help.

-- JonathanManning - 27 May 2008

-- JonathanManning - 27 May 2008

Okay I've checked these now.

1. My apache MaxRequestsPerChild was set to 4000. I reduced it to 1000, then 500. Thought it worked for a bit, but in fact it hasn't.

2. My WWW::Mechanize works with a single session- I know it re-sends the cookie, since I only authenticate once at the start of the script.

3. No, I do not run parallel requests- though if I work this out maybe I will start doing so....

So the head-scratching continues.......

-- JonathanManning - 27 May 2008

On second reading I see that I wrote HowToTrackDownCircularReferences back in 2006. Maybe the warning "This recipe works for releases based on TWikiRelease04x00 only!" actually means that you need at least version 4.0, and you might give it a try. In Bugs:Item2158, SvenDowideit also mentions a utility in the tools directory which can be grabbed from the SVN repository. Maybe this tool, or at least the pointer to Devel::Monitor used by this tool, can aid?

-- HaraldJoerg - 27 May 2008

Thanks for the advice- I tried the instructions in that topic. I found a circular reference in the view script which was fixed by adding %$this = (); to the TWiki.pm finish subroutine. But I didn't find this for the save script, despite clearly escalating memory requirements (viewed with top) of a script like:

#!/usr/bin/perl -w
my $save  =  '';
my $n_loops  =  $ARGV[0] || 100;
my $i_loops  =  0;
{
    open SAVE, './save' or die "Couldn't open save script: '$!'";
    local $/ = undef;
    $save = <SAVE>;
    close SAVE;
}

eval "sub handler { $save }";
die "Couldn't compile handler: '$@'" if $@;

while ($i_loops++ < $n_loops) {
    handler();
}

And problems persisted even with the ammendment to the finish subroutine. I'm afraid I'm out of my depth a little here- but I don't know what's going on.

-- JonathanManning - 28 May 2008

Okay, I'm wimping out. I have disabled the mod_perl for the save script for now during my buk calls to save.

-- JonathanManning - 29 May 2008

Closing this after more than 30 days of inactivity. Please feel free to re-open if needed.

-- PeterThoeny - 02 Aug 2008

Re-opening this since I see the same issue here on twiki.org after upgrading to TWiki 4.2.2 and using mod_perl. After two days the server crashes (does not respond). There are many "Can't fork" messages indicating a memory leak:

[Sat Aug 30 00:38:14 2008] [error] [Sat Aug 30 00:38:14 2008] -e: Can't fork at /var/www/twiki-local/lib/TWiki/Sandbox.pm line 381.

For now as a workaround I restart Apache once evry 6 hours.

-- PeterThoeny - 31 Aug 2008

I run several servers that use mod_perl, and haven't had any of the problems described here. The main difference perhaps is that I only use mod_perl for view and viewfile. Traffic on the other scripts is low enough that the advantages of mod_perl are slight.

-- CrawfordCurrie - 31 Aug 2008

Kenneth disabled mod_perl on twiki.org. This is the right move, a trade-off between page access time (1.2 sec vs 1.4 sec) and stability.

-- PeterThoeny - 01 Sep 2008