Question

Below is a search that I copied from somewhere. However, why it works, I don't understand. In particular, the first field defining the query is a mystery to me. Anybody who can shed light on what is happening here? Is there a way of adding explanations of this to the documentation?

%SEARCH{ "[S]tatus.*(td..td|value\=).*[W]aiting" casesensitive="on" regex="on" 
nosearch="on" nototal="on" format="| [[$topic]]<br /> ($date - $rev - 
[[%SCRIPTURL%/rdiff%SCRIPTSUFFIX%/$web/$topic][Diffs]]) |"}%

Another related question.... Is there a way of making searches like "for all the topics that have a form X, if the field Y is bla bla bla,..."

It would be great to get some feel for what the limits of the search capability are.

• TWiki version: Athens
• Web server: Apache
• Server OS: All
• Web browser: IE
• Client OS: All

-- ThomasWeigert - 01 Aug 2002

Answer

I've taken a stab at this, but since I'm fairly new to regexes, someone might correct me. After an answer is fleshed out a little better I'll probably move a copy to WikiLearn -- see Wikilearn.RegularExpression and Wikilearn.RegexRegularExpressionsExplained. If anybody sees errors on those pages, feel free to fix them.

The first field, [S]tatus.*(td..td|value\).*[W]aiting= is a regular expression (regex) or search pattern. It is searching for a string (in something -- not sure what -- I guess within a page (topic) for:

• The word "Status"
• followed by anything (.* -- or more precisely, 0 or more instances of anything -- the period matches anything, and the * indicates 0 or more repetitions of that match)
• followed by, hmmm, not sure what the td..td construct is -- it doesn't immediately seem meaningful to me so I'm guessing it's a special built-in search pattern in Perl -- UPDATE: Nope, and PeterThoeny has a better explanation below. He also explains how this whole search pattern is designed to look for either (current) form fields or (old) categories. My explanation here should be refactored based on the information from Peter. I'm not ready to to that now, though.
• or "value=" (the = sign would have some special meaning in a regex, but it is escaped by the \, so it is treated as a real "="
• followed by anything (.* -- or more precisely, 0 or more instances of anything -- the period matches anything, and the * indicates 0 or more repetitions of that match)
• followed by the word "Waiting"

Putting the S and W in square brackets is a special trick so that the search does not find the search. In other words, so it does not find the string:

"[S]tatus.*(td..td|value\=).*[W]aiting"

on the TWiki page containing the search command:

% SEARCH{ "[S]tatus.*(td..td|value\=).*[W]aiting" casesensitive="on" regex="on" 
nosearch="on" nototal="on" format="| [[$topic]]
 ($date - $rev - 
[[https://twiki.org/cgi-bin/rdiff/$web/$topic][Diffs]]) |"}%

The search string could have been written like this and get almost the same results, the only difference being it would also find itself:

% SEARCH{ "Status.*(td..td|value\=).*Waiting" casesensitive="on" regex="on" nosearch="on" nototal="on" format="| [[$topic]]
 ($date - $rev - 
[[https://twiki.org/cgi-bin/rdiff/$web/$topic][Diffs]]) |"}%

-- RandyKramer - 02 Aug 2002

Thanks. Actually, I was not wondering about the regular expression part.... What I was struggling with was to understand

• why is this searching the metadata
• what is the meaning of the td..td

Maybe I am misunderstanding what the search actually searches. I was under the assumption that %SEARCH looks only through the text, not the metadata, as there is %METASEARCH.

-- ThomasWeigert - 02 Aug 2002

%SEARCH depends on grep, and grep searches the whole file, including the meta data.

An example meta data form field is:

%META:FIELD{name="OperatingSystem" title="OperatingSystem" value="OsWin"}%

So a search for a form field could look like:

%SEARCH{ "[O]peratingSystem.*value\=.*[O]sWin" regex="on" ... }%

Using square brackets is a trick to avoid a hit on the topic doing the search.

Now the original file format of the category table (the predecessor of the TWiki forms) looks like this:

<td valign="top" align="right"> OperatingSystem:  <br></td><td>  OsWin </td>

The following search finds topics in the old and new format:

%SEARCH{ "[O]peratingSystem.*(td..td|value\=).*[O]sWin" regex="on" ... }%

The td..td matches td<>td; a simple search on "[O]peratingSystem.*[O]sWin" could find a hit in the topic text by coincidence.

A simple %SEARCH{ "[O]peratingSystem.*value\=.*[O]sWin" ...}% search is sufficient if you do not have topics in the old format.

Regarding searches like "for all the topics that have a form X, if the field Y is bla bla bla,...": Do an AND search like

%SEARCH{ "META\:FORM.*name\=.*WebForm;[O]peratingSystem.*value\=.*[O]sWin" regex="on" ... }%

-- PeterThoeny - 02 Aug 2002

Added an example of metadata (Peter apparently forgot -- oops, maybe he didn't -- until I added the <nop> it seemed like the metadata example would just disappear when the page is saved (or maybe on the second save?) -- anyway, hopefully the example will stay now) and some corrections to my first note based on information provided by Peter above.

If I think about it, the strange disappearance of the metadata example may be entirely consistent with the way TWiki has to work (I haven't tried thinking about it yet ).

-- RandyKramer - 02 Aug 2002

PeterThoeny, your explanation is great. The trick about the AND search is really worth putting into the documentation...

-- ThomasWeigert - 03 Aug 2002

Added most of this page to SearchPatternCookbook

-- JosMaccabiani - 03 Sep 2005

Thanks Jos!

-- PeterThoeny - 04 Sep 2005