%META:TOPICINFO{author="TWikiContributor" date="1304147514" format="1.1" version="$Rev$"}%
---+!! !NativeSearchContrib
<!--
   Contributions to this contrib are appreciated. Please update the contrib page at
   http://twiki.org/cgi-bin/view/Plugins/NativeSearchContrib or provide feedback at
   http://twiki.org/cgi-bin/view/Plugins/NativeSearchContribDev.
   If you are a TWiki contributor please update the contrib in the SVN repository.

   * Set SHORTDESCRIPTION = Native code search for use with CGI accelerators
-->
<sticky><div style="float:right; background-color:#EBEEF0; margin:0 0 20px 20px; padding: 0 10px 0 10px;">
%TOC{title="Page contents"}%
</div></sticky>
%SHORTDESCRIPTION%

---++ Introduction

By default TWiki searches its database by [[http://en.wikipedia.org/wiki/Fork_%28operating_system%29][forking]] the standard [[http://en.wikipedia.org/wiki/Grep][grep]] program in a sub-process to search through the text files in the database cache. This is usually pretty fast, but unfortunately in some environments (specifically with [[http://perl.apache.org/][mod_perl]]) forking a sub-process is very expensive, and another approach is needed.

!NativeSearch is a c-code module that can be plugged into TWiki to replace the default searching algorithm. It publishes an [[http://en.wikipedia.org/wiki/XS_(Perl)][XS]] interface that allows it to be called from Perl, without having to fork a sub-process. It has comparable performance to the standard forking search in most environments, but is often significantly faster than forked search when TWiki is run under a CGI accelerator. The effect is most pronounced with mod_perl, but is also there for [[http://www.daemoninc.com/SpeedyCGI/][SpeedyCGI]].

*You require shell access to the server to install this module*. Refer to TWiki:Codev.NativeSearch for a list of systems where the module has been successfully compiled and installed (and please update it if you find a new one).

---++ Installation Instructions

If you are installing on a TWiki version before 4.1.2, please refer to TWiki:Codev.NativeSearch for instructions. Note that an old version of the package was pre-installed with 4.1.2; that pre-installation will be overwritten by this package.

Before you start, there are some prerequisites.
   1 you must have =ExtUtils::MakeMaker= installed (available from CPAN)
   1 you must have a C compiler (=gcc= has been tested). Most Unix/Linux systems will have this already. On Win32 !MinGW is recommended.
   1 you will need =make= (a standard development tool). Unix/Linux users will probably have this already; others may have to search. You are recommended to use =nmake= on Win32. 
   1 finally you will need the [[http://www.pcre.org/][pcre]] library (required for perl compatible regular expressions). This is a standard Linux library and will be available in an RPM or Debian package or whatever for your system. It is also available from !GnuWin32 for Windows. You need both the runtime library (binary) and also the developer header files, which are usually shipped in a seperate package.

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server where TWiki is running.

Like many other TWiki extensions, this module is shipped with a fully
automatic installer script written using the Build<nop>Contrib.
   * If you have TWiki 4.2 or later, you can install from the =configure= interface (Go to Plugins->Find More Extensions)
      * See the [[http://twiki.org/cgi-bin/view/Plugins/BuildContribInstallationSupplement][installation supplement]] on TWiki.org for more information.
   * If you have any problems, then you can still install manually from the command-line:
      1 Download one of the =.zip= or =.tgz= archives
      1 Unpack the archive in the root directory of your TWiki installation.
      1 Run the installer script ( =perl &lt;module&gt;_installer= )
      1 Run =configure= and enable the module, if it is a plugin.
      1 Repeat for any missing dependencies.
   * If you are *still* having problems, then instead of running the installer script:
      1 Make sure that the file permissions allow the webserver user to access all files.
      1 Check in any installed files that have existing =,v= files in your existing install (take care *not* to lock the files when you check in)
      1 Manually edit !LocalSite.cfg to set any configuration variables.

%IF{"defined 'SYSTEMWEB'" else="<div class='twikiAlert'>%X% WARNING: SYSTEMWEB is not defined in this TWiki. Please add these definitions to your %MAINWEB%.TWikiPreferences, if they are not already there:<br><pre>   * <nop>Set SYSTEMWEB = %<nop>TWIKIWEB%<br>   * <nop>Set USERSWEB = %<nop>MAINWEB%</pre></div>"}%


---+++ Building the c-code

=cd= to the root directory in your TWiki installation. Now, _assuming you have administrator (root) access to the server:_
<verbatim>
$ cd tools/native_search
$ perl Makefile.PL
$ make install
$ perl test.pl -i -l NativeTWikiSearch test.pl Makefile.PL NativeTWikiSearch.xs
</verbatim>
(on Win32 you should use =nmake=)

Make sure that =perl test.pl= returns something sane, and _NOT_ a
=Segmentation fault=.
(=test.pl= is simply a stub interface to the native search, so it behaves like =grep=. The test above simply greps for the string "NativeSearch" in the files Makefile.PL and !NativeTWikiSearch.xs)

*If you do not have administrator (root) access to the server*, you can still
install. Follow the steps above, but use =perl Makefile.PL INSTALLBASE=/your/dir= to install to a different directory. Exactly where you install depends on your system configuration.

Now, in [[%SCRIPTURL{configure}%][configure]], under "Store Settings", you should be able to select the search algorithm =TWiki::Store::SearchAlgorithms::Native= (click the <input type="button" style="background-color:#eef" value="Yes, I've read all the documentation"/> button first if you have TWiki 4.2 or later).

Run a TWiki search, and watch the web server error log for any problems.

Because performance can differ widely between two apparently similar webserver configurations, you are recommended to do your own benchmarking and select the search algorithm that performs best for you.

---++ Contrib Info

Many thanks to the following sponsors for supporting this work:
   * Wind River Systems http://windriver.com

|  Contrib Author: | Crawford Currie http://c-dot.co.uk |
|  Copyright: | &copy; 2007 !WikiRing; <br /> &copy; 2007-2011 TWiki:TWiki/TWikiContributor |
|  License: | GPL ([[http://www.gnu.org/copyleft/gpl.html][GNU General Public License]]) |
|  Contrib Version: | 21570 (2011-07-06) |
|  Change History: | <!-- versions below in reverse order -->&nbsp; |
|  2011-07-06: | TWikibug:Item6764: Doc improvements -- TWiki:Main.PeterThoeny |
|  2010-05-03: | TWikibug:Item6433: Doc improvements; replacing TWIKIWEB with SYSTEMWEB |
|  2007-09-27: | TWikibug:Item4713: Initial version |
|  Dependencies: | <table border="1"><tr><th>Name</th><th>Version</th><th>Description</th></tr><tr><td align="left">ExtUtils::MakeMaker</td><td align="left">&gt;=0</td><td align="left">Required for building makefiles</td></tr></table> |
|  Contrib Home: | http://twiki.org/cgi-bin/view/Plugins/NativeSearchContrib |
|  Feedback: | http://twiki.org/cgi-bin/view/Plugins/NativeSearchContribDev |
|  Appraisal: | http://twiki.org/cgi-bin/view/Plugins/NativeSearchContribAppraisal |

__Related Topics:__ %SYSTEMWEB%.TWikiContribs, %SYSTEMWEB%.DeveloperDocumentationCategory, %SYSTEMWEB%.AdminDocumentationCategory, %SYSTEMWEB%.TWikiPreferences
