create new tag
, view all tags

SID-02356: Could not perform search. Error was: /bin/grep -F -i -l -H

Status: Asked Asked TWiki version: 6.0.2 Perl version: 5.26.3
Category: CategorySearch Server OS: CentOS 7, 3.10.0-693.el7.x86_64 Last update: 2 hours ago

I have recently migrated from 4.3 to 6.0.2 and converted from apache authentication to AD authentication. All that has gone great and the site works great with one exception. If I type in the search box i get the below error.

Could not perform search. Error was: /bin/grep -F -i -l -H -- %TOKEN|U% %FILES|F% Grep for 'Compliance' returned error

I searched the community and found: http://twiki.org/cgi-bin/view/Support/SID-01381 This user had a very similar issue and discovered he had some files / directories not readable by his webuser. Unfortunately, he did not share the affected files /folders.

In similar fashion as his issue, I can change the end of the below URL to Main or any WEB and it will search just fine. It just does not work when it's set to "All" or "On"


I'm using user/group apache and have scoured the folders and files and can't find any that would cause an impact or not readable by apache for some reason.

Any thoughts/ directions?

-- Christopher Crowe - 2018-04-25

Discussion and Answer

Well, permissions often are the cause for this problem. There are a few tests to this:

  • TWiki 6.0.2 has two directories in /data which aren't user accessible webs: /data/_default and /data/_empty - maybe one of those didn't exist in 4.3 and has wrong permissions now?
  • The Web server's error log might have more details on the error.
  • You could also try to change the EXPERT setting of {RCS}{SearchAlgorithm} to TWiki::Store::SearchAlgorithms::PurePerl and check whether this provides better diagnostics.

-- Harald Jörg - 2018-04-25

Hi Harald, thanks for providing some options to narrow down the issue.

Unfortunately, the web logs don't show anything related to the issue. The only log that shows an error is the TWiki log and it only repeats what I pasted above. However, changing the SearchAlgorithm to PurePerl did solve the problem as I don't run in to the same error now. If I switch it back to Fork, the problem returns. I'll do some reading on the differences between the two,going to leave it on PurePerl for now, but will continue to troubelshoot.

Since you pointed out /data/_default and /data/_empty, i compared the directories to a working copy and the file permissions are different. On the TWiki site that has the broken search, the files are 0755 and on the working site they are mostly 0644 aside from the revision files which are 0444. Going to change the permissions to match and will update the results.

-- Christopher Crowe - 2018-04-26

The permissions on the working site are fine. Execute permission is not necessary (and not desired) for files, but required for directories. However, I'd be surprised if this makes a difference.

If the pure Perl algorithm works, then I've two more wild guesses:

  • Do you have a very large number of topics, or extremely long topic names in a web? I seem to recall that the forking search had some issues with creating too long command strings in the past, but this should have been fixed by limiting to 512 topics per fork.
  • Do you have webs or topics with non-ASCII directory or file names? If so, you might be trapped in an encoding minefield between TWiki, the shell, and the file system.

Both search algorithms loop over the list of webs. I've no clue why searching individual webs works but "All" doesn't, and what could have been changed between the two versions of TWiki.

-- Harald Jörg - 2018-04-26

You are right, permission change made no difference.

1. We do have thousands of topics, but not a ton of Webs. 2. We do not have topics or webs with non-ASCII names.

I don't believe it's a # of topics issue since the working server doesn't have this issue and it's essentially a replica of the one I'm currently working on. I've confirmed it's also using Fork.

Is there a negative to using PurePerl for the search algorithm that you are aware of?

I'm also baffled as to why individual searches work but "All" does not. We have about 50 webs, i may painstakingly search one to see if any break.

-- Christopher Crowe - 2018-04-26

About using PurePerl: It might have a negative performance effect, depending on your situation. The forking algorithm uses the system's grep command (so it must fork, it does not fork to achieve parallelism) while the pure Perl algorithm does it all in one process.

I can only guess that it is one of the webs which is causing problems. Using the %SEARCH{}% function in a sandbox topic (TWiki.VarSEARCH) you can provide a list of webs for every search, this can be used speed up looking for the "bad" one.

-- Harald Jörg - 2018-04-27

      Change status to:
ALERT! If you answer a question - or someone answered one of your questions - please remember to edit the page and set the status to answered. The status selector is below the edit box.
Status Asked
Title Could not perform search. Error was: /bin/grep -F -i -l -H
SupportCategory CategorySearch
TWiki version 6.0.2
Server OS CentOS 7, 3.10.0-693.el7.x86_64
Web server Apache 2.4.6
Perl version 5.26.3
Browser & version Chrome, IE, Edge Latest
Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r6 - 2018-04-27 - HaraldJoerg
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.