mod_perl on Unix

These instructions apply to TWiki-4 only. While these instructions should work (with minor changes) with earlier TWiki releases, they have not been tested. Caveat emptor.

Getting TWiki to work with mod_perl on Unix is quite straightforward, with the caveat that some plugins may not work correctly. This guide is written for Linux, but this approach should work on other systems as well.

See also ModPerl for more general discussion.

WARNING There is a known problem with Apache 2 when some of the TWiki scripts are enabled for mod_perl, but others are not. It appears that CGI parameters can be lost, when a mixture of mod_perl and non- mod_perl scripts are used. This seems to be a bug in Apache, mod_perl or CGI. If you see "Incorrect parameters to the save script" this is an indication that you have the problem. The solution is to run all TWiki scripts (except configure) under mod_perl.

VERY IMPORTANT

TWiki will work with mod_perl 'out of the box'. However mod_perl has a very nasty problem, which means that searching in TWiki running under mod_perl has been observed to be ten times slower than normal TWiki,. This might make you wonder why we bother with mod_perl. Fortunately most sites are not heavily dependent on search performance, and can accept this slowdown for the other advantages mod_perl affords.

Fortunately there are two alternative search algorithms that can be used with TWiki; pure perl search, and NativeSearch.

• Pure perl search simply replaces the existing grep with a perl implementation. This is about 4 times slower than the standard grep implementation outside of mod_perl, and about twice as fast under mod_perl.
• NativeSearch is a c-code module that can be installed and then called by TWiki. It is very slightly slower than the standard grep outside of mod_perl, but it is just as fast under mod_perl. It requires some standard Unix development tools to install.

Other accelerators (such as SpeedyCGI) don't seem to suffer as badly from this effect, and it may be worthwhile benchmarking one or more of them on your server.

If you have TWiki 4.1.2 or later, you can install native search by following the instructions in the tools/native_search/INSTALL document. The searching algorithm is selected from the 'Store Settings' section of configure.

What platforms

• Apache/1.3.28 on Linux 2.4.21-286-athlon
• Apache 2.0.52 on Linux 2.6.9-22.EL
• Apache/2.0.55 on Linux Debian unstable

Prerequisites

• You have to have mod_perl installed on your Apache server, and it must be available to the TWiki scripts. Instructions for installing mod_perl are outside the scope of this guide.
  • NOTE: with mod_perl, your apache will use a lot more memory, so you will need to tune your apache configuration to avoid running out.
• You have to be able to edit the Apache configuration file, httpd.conf.
• You must be using an absolute path in bin/LocalLib.cfg, as described in the installation instructions.

You may use ApacheConfigGenerator to try and generate a suitable apache config file, provided that you activate the "Enable mod_perl (expert setting)" checkbox in the config generator's form.

Enabling mod_perl

First you have to explain to mod_perl where to find the TWiki libraries. This has to be done in httpd.conf.

In httpd.conf add this:

Apache 1.3

   PerlRequire /usr/local/twiki/tools/mod_perl_startup.pl
   # Turn on taint checking (only add line below if it is not already in the httpd.conf)   
   PerlTaintCheck On

Apache 2

   PerlRequire /usr/local/twiki/tools/mod_perl_startup.pl
   # Turn on taint checking
   PerlSwitches -T

Replacing /usr/local/twiki/ with the file path to your TWiki installation, as seen by the webserver.

Note if you can't write to httpd.conf then it is probable that there is some other way for you to run a perl startup script. Check your webserver configuration documents. Many distributions have a directory from which the httpd.conf file includes any file with the statement Include conf.d/*.conf
The include file that controls mod_perl is often called perl.conf. This is then where the PerlSwitches -T would be found. Same file also contains the command that loads the mod_perl module. In Apache2 this is LoadModule perl_module modules/mod_perl.so which must be run before you call PerlRequire and PerlSwitches.

Apache 1.3 note While you are in httpd.conf, check to see if httpd.conf contains the command PerlTaintCheck On. If it doesn't, you should add it. If that breaks other (non-TWiki) Perl scripts already running under mod_perl, then you should really fix those scripts! (CGI perl scripts should always run with the T switch enabled). If you can't do that, then as a last resort you can remove the T (taint) option from the perl command on the first line of the TWiki scripts being run under mod_perl.

Now create tools/mod_perl_startup.pl under your TWiki installation (or edit your existing mod_perl startup file). It should contain the following:

$ENV{MOD_PERL} =~ /mod_perl/ or die "mod_perl_startup called, but mod_perl not used!";
use lib qw( /usr/local/twiki/bin );
require 'setlib.cfg';
1;

Make sure you have a twiki/bin/LocalLib.cfg file with $twikiLibPath correctly set to the absolute path of the twiki/lib directory.

Now, when the webserver starts up, the TWiki libraries should be added to the @INC path.

To enable mod_perl on all of the TWiki scripts, create (or edit) bin/.htaccess and add the following. All scripts except configure should be run under mod_perl.

# This matches all lower-case file names, except
# those starting with "configure"
<FilesMatch "^(?!configure)[a-z.]+$">
   SetHandler perl-script
   PerlHandler Apache::Registry
</FilesMatch>

# This matches all lower-case file names, except
# those starting with "configure"
<FilesMatch "^(?!configure)[a-z.]+$">
   SetHandler perl-script
   PerlResponseHandler ModPerl::Registry
   PerlSendHeader On
   PerlOptions +ParseHeaders
</FilesMatch>

(Note that Apache will use mod_perl for all matching files. So if you have other files types, such as graphics files, in your bin directory, you might get warning messages such as Unrecognized character \x89. Some poorly-written plugins may install such files. You can overcome this by explicitly listing script filenames in the FilesMatch regular expression e.g. FilesMatch "^(attach|changes|.... See the Apache documentation on FilesMatch for more information).

Note if you set up TWiki in httpd.conf and don't want to use .htaccess, then you can add the same lines to the <Directory> section for the bin dir in httpd.conf.

Restart the webserver, and you should be hot to trot.

What to do if it doesn't work

• Remember that if you change your configuration or modify any scripts, you need to restart the Apache server to make sure you are working from a clean slate.
• If TWiki is complaining that it can't find a code module, it may be because your path settings are wrong. Check that your PerlRequire statement is correct.
• If TWiki seems to be mostly working but you are seeing strange rendering effects, then try disabling any non-default plugins. If the problem disappears, re-add the plugins one by one until you have isolated the problem.
  • If you have problems with plugins (many have not been written with mod_perl in mind) you may be able to improve things by only enabling mod_perl on a subset of the scripts. Usually the view and viewfile scripts are the most visited on TWiki sites, and respond best to acceleration. Beware of the Apache 2 problem described at the top of this page, though! See also HowToWriteModPerlCompliantPlugins.
• If you know about Apache::DB, be warned. TWiki is a large application, and running with Apache::DB enabled is not recommended.
• At any point in the perl scripts you can print to STDERR from scripts to aid in debugging. The output will be printed to the Apache error_log.
• If you just can't get it to work, consider using a different accelerator. speedycgi has been found to be easier to get working reliably, and is almost as fast.
• A lot of older material has been removed from this topic; if you are researching a problem, and are really desperate, you might try looking back at previous versions.
• for some interesting info install Apache::Status or Apache2::Status and goto http://localhost/perl-status

How to write mod_perl compliant extensions

-- Contributors: CrawfordCurrie, KennethLavrsen, DavidStults, SvenDowideit, StefanSchmidbauer, HaraldJoerg, JaredRhine

Discussions

I was able to fix this problem for my twiki without installing mod_perl.

Environment: Dakar release 4x00x00, Apache 1.3.26, perl 5.6.1, Red Hat Linux 6.1

Fix: /twiki/lib/TWiki/Search.pm, line 36:

- my $emptySearch = 'something.Very/unLikelyTo+search-for;-)';

+ my $emptySearch = 'something.Very/unLikelyTo+search-for;-()';

-- ElizabethAuden - 27 Apr 2006

AFAICT this has no relevance to this cookbook; were you perhaps meaning to post to an entry in Support web?

-- CrawfordCurrie - 28 Apr 2006

Unfortunately, I "just can't get it to work". Are there certain versions of mod_perl that are verboten? I'm using mod_perl 2.0.2.

-- JohnPaulPagano - 03 Jun 2006

Very good article, TWiki feels so much faster now.

-- StephaneLenclud - 17 Sep 2006

Instruction didn't work to me, since the startup-script could not locate "lib.pm" in @INC. Figured it was a principal problem that @INC is FROZEN before the script starts, so that you have to set in your httpd.conf:

SetEnv PERL5LIB /your/path/to/twiki/lib

IMPORTANT: Setting PERL5LIB requires not to use the directive PerlRequire; no startup-script is required. Requiring this script issues just another "could not locate lib.pm." So following the instructions does not (not necessarily always) work; there are reasons for it (http://blogs.sun.com/alanbur/entry/twiki_rant).

-- JoergVolbers - 28 Sep 2006

Sorry, there are a couple of mistakes. If your startup script can't locate lib.pm, then the problem is deeper. lib.pm is one of Perl's standard pragmas, the path is compiled into Perl. It should be available in every Perl installation, regardless of mod_perl or TWiki. I know of two possible sources of such problems, though there might be more:

1. Your webserver is running in a chroot environment, so the paths it sees are different from the paths at Perl's installation time
2. Someone explicitly replaced @INC instead of adding to it.

Setting PERL5LIB does not work if you have taint checking enabled, as you'll find in the perlrun manpage. TWiki usually enables taint checking with a -T switch in the shebang line, but with mod_perl this is too late, that's why mod_perl needs either PerlTaintCheck On or PerlSwitches -T, depending on your version of Apache.

Running a startup script is an excellent idea in mod_perl in general, and TWiki in particular, since in a startup script you can pre-compile all modules from TWiki core and all your plugins before a user requests one of your pages.

Alan Burlison's rant is just that: A rant. It is known in the developer community, and if you look close you'll notice that this guide has been improved since he wrote his article. But if you look closely, you'll note that he started from false assumptions and from there worked downward by looking into TWiki code instead of mod_perl documentation.

There is nothing like a frozen @INC in Perl, mod_perl, nor TWiki. All you need to know that Perl knows about compile time, and about runtime. Modifying @INC at runtime is too late to affect compilation: That's what is usually meant when people talk about "frozen" @INC. Nevertheless there are mechanisms to modify @INC:

1. Modify @INC in a BEGIN block: BEGIN blocks are executed during compilation time, so "just in time" to allow subsequent use of modules from more places.
2. Modify @INC indirectly, by saying use lib ...;. The use statement is executed during compile time, so any directory you add with use lib is available for subsequent use statements.
3. Delay compilation until the execution phase by either replacing use by require, or by wrapping it into eval{...} blocks.

So please don't jump to conclusions. Describe your problem as accurate as possible in the Support web. There are plenty of people (including me) who managed to set up TWiki with mod_perl under varying operating systems.

-- HaraldJoerg - 28 Sep 2006

I'm having a problem when running several TWiki installation on the same apache2 server in different virtual host. It works fine if only one TWiki installation has mod_perl enabled. Yet if I enable mod_perl in both, one of the installation is showing the pages form the other one. Any advice?

-- StephaneLenclud - 12 Nov 2006

Stephane, I'm afraid you are out of luck with that problem. With mod_perl, TWiki is compiled only once - but compiling TWiki includes the process of reading the configuration files. Only one set of TWiki configuration settings can be active in a mod_perl environment, because they share the same Perl namespace.

A hackish attempt would be to insert a do '../lib/LocalSite.cfg'; into all the scripts under each of the respective bin directories, to enforce that the configuration is re-read for every request. However, I doubt that this would catch all TWiki data structures which are initialized "once and for all times" in TWiki and its plugins.

-- HaraldJoerg - 12 Nov 2006

That's what I was afraid of Nevermind I guess I can live with only one virtual host mod_perled. Thanks for your answer. Could it be possible to fix that in future TWiki release or is there just no way around? Something like using different Perl namespace for each installation for instance?

-- StephaneLenclud - 13 Nov 2006

There have been some ideas how to solve this, at least in IRC, but I'd expect a long phase of testing until there's enough confidence that the TWikis are really separated in all their data (including plugin data). I wouldn't bet that some developer has it high enough on his priority list, it doesn't seem to be commonly required.

-- HaraldJoerg - 15 Nov 2006

I have a problem with gif ang png graphics files

My httpd.conf

...
<Location /perl-bin>
    SetHandler perl-script
    PerlResponseHandler ModPerl::Registry
    Options +ExecCGI
    PerlSendHeader On
    Order deny,allow
    Deny from all
    Allow from 213.134.178.186
    PerlOptions +ParseHeaders
</Location>
.....

Unrecognized character \x89

please help

-- RockyAbadon - 25 Jan 2007

After following the instructions (on Redhat) I got this error : "Can't locate Apache.pm in @INC" (etcetera). I could solve this by including the following line to tools/mod_perl_startup.pl:

use Apache::compat ();

Configuring mod_perl for Red Hat 8.0 and Apache 2.0

-- ArthurClemens - 31 Mar 2007

Has anyone tried configuring mod_operl on Ubuntu Server 7.04 (Fiesty Fawn) running twiki 4.1.2 release and Apache 2.2.3? I tried just like its said in this doc but when i restart apache2 iget

Syntax error on line 1 of /etc/apache2/httpd.conf:
Invalid command 'PerlRequire', perhaps misspelled or defined by a module not included in the server configuration
   ...fail!

-- SibiJoseph - 27 Apr 2007

Have you installed mod_perl?

-- AndreaSterbini - 09 Jun 2007

Just a note that the above instructions worked a treat for OSX 10.4, with the exception of the "Filesmatch" regexp which apache didn't like. I just got rid of it, as I don't routinely use config (and it warns you if you try). TWiki 4.0.4

-- PiersGoodhew - 18 Jul 2007

About two months ago I started TWikiStandAlone project and I've just released its first version. There is native support for ModPerl, versions 1.x, 1.9 and 2.0+. There is also support for other execution modes. One of my goals is that whatever execution mode is chosen, it should be easy to use. See mod_perl configuration tips.

-- GilmarSantosJr - 29 Oct 2007

Actually you don't need the mod_perl-startup file, just write

PerlRequire /path/to/twiki/bin/setlib.cfg

I can't make the TWikiStandAlone configuration work though, where TWiki::UI is the handler. Somehow the library paths imported by setlib.cfg aren't active when trying to load TWiki::UI...

-- WoutMertens - 27 Feb 2008

Wout: With only PerlRequire, it can't work. setlib.cfg usually does not explicitly define library paths, it calls LocalLib.cfg to do so. However, lacking the path to LocalLib.cfg this fails, and setlib.cfg handles this failure by using default, relative paths. However, relative paths don't work with mod_perl because it doesn't start in TWiki's bin directory.

Having a mod_perl startup file is actually a very good idea, because you can define your libraries and use your modules from there, making sure that all the compilation takes place in shared memory, before forking takes place. You just need to define your library paths, which may be different from Gilmar's.

-- HaraldJoerg - 27 Feb 2008

Harald is quite right, by listing as many of the libraries TWiki needs in the mod_perl startup, you are making sure that the subsequent requests do less work.

-- SvenDowideit - 28 Feb 2008

Note that with the latest versions of Mod_Perl 2, PerlSendHeader has been removed and will cause syntax errors. It is replaced by the PerlOptions +ParseHeaders statement.

-- GeorgeClark - 30 Aug 2008

I'm using a TWiki 4.3 with Apache 2.2.17 and mod_perl2 and I experienced other time differences.

Without mod_perl the Forking and the PurePerl search algos need about the same time (so no differences). And with mod_perl and the PurePerl algo normal pages are a lot faster than without mod_perl and complex searches are just a hint slower (they definitely don't need twice as long!).

-- CarlaReitzenstein - 2011-09-28