It has long been a source of considerable irritation to me that plugins have to manage their own searches over webs; the only way available for them to search in webs is to cook up some horrific %SEARCH expression and then expandCommonVariables, followed by a hairy parse of the result. Hardly the most elegant approach.

While refactoring the core I found I could cleanly move the main bit of the search into Store, out of SearchDotPm (thanks to Peter's organised design of Search it was fairly easy).

While upgrading the ActionTrackerPlugin I started investigating whether that search function could be used. Indeed it can, and it significantly simplifies the code in the plugin! So I am proposing that the following function be added to func (this is merely a stub to the actual function in the Store module):

searchInWebContent($searchString, $web, \@topics, \%options ) -> \%map

Search for a string in the content of a web. The search is over all content, including meta-data. Meta-data matches will be returned as formatted lines within the topic content (meta-data matches are returned as lines of the format %META:\w+{.*}%)

• $searchString - the search string, in egrep format
• $web - The web to search in
• \@topics - reference to a list of topics to search
• \%option - reference to an options hash

The \%options hash may contain the following options:

• type - if regex will perform a egrep-syntax RE search (default '')
• casesensitive - false to ignore case (defaulkt true)
• files_without_match - true to return files only (default false). If files_without_match is specified, it will return on the first match in each topic (i.e. it will return only one match per topic, and will not return matching lines).

The return value is a reference to a hash which maps each matching topic name to a list of the lines in that topic that matched the search, as would be returned by 'grep'.

To iterate over the returned topics use:

my $result = TWiki::Func::searchInWebContent( "Slimy Toad", $web, \@topics,
   { casesensitive => 0, files_without_match => 0 } );
foreach my $topic (keys %$result ) {
   foreach my $matching_line ( @{$result->{$topic}} ) {
      ...etc

---

This is currently documented in TWikiFuncModule: Recommended to use TWiki::Func::expandCommonVariables( "%SEARCH{...}%" );. Not pretty but functional.

As you suggested, I believe it is useful to explicitely add a search function. I find the proposed ( $searchString, $web, $type, $caseSensitive, $justTopics, \@topics ) parameters somewhat limiting, and the interface needs to be changed with each enhancement to SEARCH. How about simply using the SEARCH parameter interface? That has the advantage that it is well documented and is exensible, e.g. new functionality is made availabe with each release without the need to add/change parameters.

That is, the function interface can be as simple as:

searchText( %params ) -> $text

with %params containing any of the valid parameters documented in TWikiSearch. The SEARCH parameters should be enhanced evolutionary with compatibility in mind, e.g. we should not face compatibility issues in the future.

-- PeterThoeny - 30 Mar 2005

searchText( \%params ) -> $text

-- WillNorris - 30 Mar 2005

I really don't mind; I stuck with what was the existing interface in Search.pm when i moved the function. Using a param hash is fine by me. I checked in the existing impl, but will change it shortly (it's currently only used in the action tracker)

-- CrawfordCurrie - 30 Mar 2005

param hash reference ?

-- WillNorris - 30 Mar 2005

OK, I changed it to the spec shown above.

-- CrawfordCurrie - 30 Mar 2005

Passing a hash or hash reference does not really matter (search does not change the hash content).

Crawford, what I meant was that it is better to support any parameter the existing TWikiSearch offers, now and in the future. That way we do not need to change the interface and the documentation with new features to SEARCH.

So, how about this interface:

searchText( $searchString, \%options ) ==> $text

Description:	Search for a string in topic name or topic text. All the options of the TWikiSearch may be used. The search is done over all content, including meta-data.
Parameter: $searchString	Search string, literal or regular expression, depending on options
Parameter: \%options	reference to an options hash. It may contain any of the parameters defined in TWikiSearch (except for search which is the $searchString parameter)
Return: $text	Formatted search output, depending on options
Since:	TWiki::Plugins::VERSION ???

Example:

my $result = TWiki::Func::searchText( "Slimy Toad", { web => $web, casesensitive => 'on',
     nosearch => 'on', nototal => 'on', format => '$topic', separator => ',' } );
foreach my $topic (split( ',' $result ) ) {
    # do something
}

With the format => '...' it is possible to do any FormattedSearch the Plugin might need.

With a multipe => on it is possible to return more then one hit per topic.

-- PeterThoeny - 31 Mar 2005

can you please, please please, stop returning $text? its totally useless for anything other than rendered output, which has absolutly NO place in Store. (and for that matter, nor do the format, seperator parameters belong in this function, rather they need to be up in Render::)

-- SvenDowideit - 01 Apr 2005

Sven is reacting to the fact that this function is simply a stub on the StoreDotPm function. Of course, it could do a lot more and be an interface to the main formatted search method, but I'm not sure what the point of that would be:

1. As you previously pointed out, you can already do that with expandCommonVariables("%SEARCH{...
2. It defeats the object of not needing to parse search results in the plugin.
3. Rendering search results from a search in a plugin implies a recursive call to several of the plugins handlers. I don't want to have to think about that when I'm writing a plugin.
4. The results of the search would be rendered and are therefore not exactly what is in the topic, making it much harder to match the same text again in the topic text (e.g to recover context)

Basically I don't want a formatted search; I want an unformatted search. This function provides that. Note that Search.pm is implemented to use this function (or rather, the function in Store for which this is a stub).

-- CrawfordCurrie - 01 Apr 2005