Bug: WebIndex Does Not Work

Clicking on Index in the topic list (link to WebIndex page) does not show any matches, even though topics exist. Advanced search on topic doesn't work either. It seems to be a problem in search.pm where the output of ls is piped to egrep. Turned on debug and confirmed that no results are returned from this command. See line 200 in search.pm "$cmd = "$TWiki::lsCmd *.txt | ...."

I am using cygwin 2.125.2.5. I have c:\cygwin\bin in the start of the path. TWiki.cfg contains these entries:

$lsCmd = "ls";
$egrepCmd = "egrep";
$fgrepCmd = "fgrep";

I have tried every imaginable combination of paths, extensions etc.

Simple test scripts work fine, so I am at a complete loss. This is driving me nuts. Twiki + web server + perl + cygwin is intensely fiddley and time consuming to set up. I think most people that are "just looking" would give up in 20 minutes. I have spent hours, and it is still not working right. I stuck it out this far because twiki offers the most out of the wiki implementations I have seen so far.

I never thought of myself as a masochist, but I am now undergoing a personal re-evaluation

Test case

This script produces results when called as a CGI script. No clue why similar code in search.pm does not return anything:

#! /usr/bin/perl -wT
use CWD;
chdir('/usr/website/twiki/data/Know');
print getcd . "\n";    # make sure we really have changed dir
$cmd = 'ls *.txt | egrep -i "\.*";
$tempVal = `$cmd`;
print $tempVal;

Environment

-- JimEggleston - 26 Nov 2001

Fix record

When you say you have tried every combination, have you tried putting the full path names, e.g. c:/cygwin/bin/ls.exe into the lsCmd setting?

-- MartinCleaver - 26 Nov 2001

I had the same problem. Solved it at least for apache. My setup is Win2K-Server, Apache, cygwin, Activestate-Perl.

• for apache, please make sure that you set the environment variables (PATH, TIMEZONE, TMP, etc.) in the http.cfg file as mentioned in the install instructions for Windows. do not put the full path name into the lsCmd, just make it ls.
• i also had access restrictions switched on. for that case you have to set the search cgi-script to require valid-user in bin/.htaccess

-- SteffenEichenberg - 26 Nov 2001

Yes, tried c:/cygwin/bin/ls, /cygwin/bin/ls, c:/cygwin/bin/ls.exe. There are no .htaccess files in xitami as such. I have not implemented the corresponding .aut files. The fact that my test script works as a cgi script proves that the xitami service can see and execute ls and egrep. Note that full text searches do work. It is only topic searches that have a problem.

As a test, I edited search.pm, changing line 200 from:

$cmd = "$TWiki::lsCmd *.txt | %GREP% %SWITCHES% $TWiki::cmdQuote$theSearchVal$TWiki::cmdQuote";
to:
$cmd = "$TWiki::lsCmd *.txt";

-- JimEggleston - 27 Nov 2001

Found the problem - $safeEnvPath in TWiki.cfg. The setting was:

$safeEnvPath      = "c:\\usr\\bin";

$safeEnvPath      = "c:\\usr\\bin;c:\\winnt\system32";=

It appears that piping relies on something in the system32 dir, perhaps cmd.exe.

I'll reread the Window installation doco later today, and if there is no mention of this, I will add a comment to the appropriate page at twiki.org (probably TWikiOnWindowsUsingXitami)

-- JimEggleston - 28 Nov 2001

You may have a weird value in the COMSPEC environment variable perhaps - ActivePerl always uses cmd.exe /x/c to run commands, and will not normally be affected by the PATH. Try CookbookActivePerlSetup, which lets you use the Cygwin bash shell to run commands such as ls etc.

For any future installs, try WindowsInstallCookbook, which is a much smoother installation process according to the two people that I know have tested it - you end up with a fully working TWiki without problems like this, and with working RCS and email notification.

-- RichardDonkin - 28 Feb 2002

I've got Windows 2000 + Apache + Cygwin + ActivePerl. Moving over to ActivePerl caused this problem but my fix was quite different! Search.pm creates a fine-looking command string ( ls *.txt | egrep ... ). RichardDonkin's CookbookActivePerlSetup causes this to be executed through bash -c so the result is something like this:

bash -c ls *.txt | egrep -i "\.*"

Well, that's fine, except that the pipe is handled by cmd.exe instead of bash! And because egrep is implemented as a link in cygwin, the egrep part fails when cmd.exe tries to execute it (see Apache's error log if you don't believe me). I suppose I could have changed Search.pm to quote the whole command ... ewwww.

My solution was to change $egrepCmd and $fgrepCmd in TWiki.cfg to "grep -E" and "grep -F" respectively. Then it doesn't matter whether bash or cmd.exe runs the command (assuming cygwin/bin is in Windows' PATH).

-- GladeDiviney - 21 Jun 2002

I've updated WindowsInstallCookbook to use grep -E for egrep, and similarly for fgrep. This is necessary for anyone with a fairly recent Cygwin (since Cygwin started using symbolic links for egrep and fgrep, and these links only work for Cygwin processes such as bash). This should make life easier for ActivePerl users who use Cygwin for grep.

Interesting point about bash not handling the pipe... Really the TWiki code should use a different form of the system call, ensuring that we call bash and then pass its arguments separately.

-- RichardDonkin - 20 Jul 2002