SID-00562: Search fails get error: “exec failed: Arg list too long grep

Status:	Answered	TWiki version:	4.3.2	Perl version:	5.10.1
Category:	CategorySearch	Server OS:	AIX	Last update:	16 years ago

Hi, we are getting the following error when searching: “Could not perform search. Error was: exec failed: Arg list too long grep” This error occurs on most types of searches, but some still work such as [Listing topics that start with a single letter] or [Listing all topics in the TWiki web in alphabetical order]

Since this is an AIX, we have replaced the AIX grep and instead installed the GNU grep version 2.5.1 in our freeware path: /opt/freeware/bin/

/usr/bin $ ls -l grep

lrwxrwxrwx 1 root system 28 Oct 9 11:05 egrep -> ../../opt/freeware/bin/egrep
lrwxrwxrwx 1 root system 28 Oct 9 11:05 fgrep -> ../../opt/freeware/bin/fgrep
lrwxrwxrwx 1 root system 27 Oct 9 11:05 grep -> ../../opt/freeware/bin/grep

We are sure it is the GNU grep since running grep –help indicates it is gnu. What we have tried in all combinations: – None worked
- Putting the /opt/freeware/bin/ in the SafeEnvPath
- Putting the full path in the egrep/fgrep EXPERT settings
e.g. for fgrep: /opt/freeware/bin/grep -F %CS{|-i}% %DET{|-l}% -H -- %TOKEN|U% %FILES|F%
or even: /opt/freeware/bin/fgrep %CS{|-i}% %DET{|-l}% -H -- %TOKEN|U% %FILES|F%
- searched the system to verify no other grep’s were present and could not find any.

Still we get the above Error when searching. or bringing up topics that contain searches like

Interestingly, if we break the SafeEnvPath by putting extra periods in it, the main Search WORKS. Breaking the SafeEnvPath also returns a blank (in the pages main content area) TWiki/UserToolsCategory topic, but no errors are presented.

-- GregNeugebauer - 2009-10-09

Discussion and Answer

TWiki uses grep for search. It searches in sets of topics to avoid the command line size limit. AIX might have a lower limit than Lnux or newer versions of Windows.

If you are using the default {RCS}{SearchAlgorithm} configure setting of TWiki::Store::SearchAlgorithms::Forking, you can edit twiki/lib/TWiki/Store/SearchAlgorithms/Forking.pm and change $maxTopicsInSet to a lower number, such as 128.

-- PeterThoeny - 2009-10-09

Codev.TWikiOnAIX is updated.

Yes, that is it, better yet I changed the AIX system settings:

(AIX) To Increase the Maximum Length of the Command Line

The ncargs attribute determines maximum length of the command line, including environment variables. On the AIX operating system, the default value of the ncargs attribute is four, 4–Kbyte blocks. To ensure that Application Server commands do not exceed the maximum length of the command line , increase this value to 16 4–Kbyte blocks.

Note -

After the value of the ncargs attribute is changed, no reboot or refresh of daemons is required.

1. Determine the value of the ncargs attribute.
   lsattr -EH -l sys0 | grep ncargs
2. If the value of the ncargs attribute is less than 16 4–Kbyte blocks, increase the value to 16.
   chdev -l sys0 -a ncargs=16

-- GregNeugebauer - 2009-10-09

Thanks for updating TWikiOnAIX! Yes, for performance, better to increase max command line length at the OS level.

-- PeterThoeny - 2009-10-09

If you answer a question - or someone answered one of your questions - please remember to edit the page and set the status to answered. The status selector is below the edit box.