Question

We are using TWikiForms and FormattedSearch to track various things, similar to TWiki.org's TWikiInstallation list.

When adding a new field to the form, it becomes difficult to search over the new fields because they do not yet exist on all topics that use that form. Therefore, you end up only searching through topics that include the new field.

Unless it's possible to write our search in a way to avoid this problem, is there any suggested workaround for this? For example, I could imagine a command line script that edits each topic (which should add the new field).

Many thanks, Devin

Environment

-- DevinBougie - 15 May 2008

Answer

If you answer a question - or someone answered one of your questions - please remember to edit the page and set the status to answered. The status selector is below the edit box.

With TWiki 4.2 you can use TWiki:Twiki.QuerySeach to search for topics with the form, regardless of whether the field is populated. Would that help?

Otherwise, the next time each topic is edited, the new field will be populated. Until then, there probably isn't any point in including it in your search results, since it won't have any data.

But if you need it anyway, PeterThoeny's answer on a somewhat related question was to look into the TWiki:Plugins.GlobalReplacePlugin.

-- SeanCMorgan - 15 May 2008

Devin is right that there is an issue if you are adding a field later on because you will get a mix of pages that have/don't have the new field. This makes it more difficult to query for that field because you would exclude all pages that do not yet have that field defined.

Here is a fix for this dilemma: The idea is to apply the search conditionally, depending on a URL param set to a non-empty value. Here is an example that queries for two form fields, one called "Exist", the other one "Xtra". The latter one was added later, e.g. is queried conditionally.

%SEARCH{
"META:FIELD.*?\"Exist\".*value.*?%URLPARAM{"exist"}%%IF{ "$ xtra != '' and defined xtra" then=";META:FIELD.*?\\"Xtra\\".*value.*?%URLPARAM{"xtra"}%" }%"
...}%

Interactive test with escaped SEARCH; URL param xtra is: not set, set but empty, set to 'foo'

Result:
%SEARCH{ "META:FIELD.*?\"Exist\".*value.*?" ...}%

-- PeterThoeny - 16 May 2008

Many thanks, Peter! I've tried modifying my search as you describe, and it no longer excludes pages that do not contain the new field. However, it does not seem to actually search and filter my results.

Here is the form definition of our new field:

| RecDate | date | 10 | | Rec'd Date |

Our search form contains the following input field:

<input type="date" name="qRDT" value="%URLPARAM{"qRDT"}%" size="10" />

And our search query now contains the following:

%IF{ "$ qRDT != '' and defined qRDT" then=";META:FIELD.*?\"RecDate\".*value.*?%URLPARAM{"qRDT"}%" }%

However, this just gives a list of all pages, not the ones that match our search.

I would greatly appreciate any more suggestions you might have. Of course, please let me know if there is any other information I can provide.

Also just to clarify, will the QuerySearch in 4.2 avoid this problem by searching topics regardless of whether the field exists in that topic (which, depending on interpretation, could be different than whether or not the field is populated)?

Thanks again.

-- DevinBougie - 19 May 2008

Ah, there was a bug in my example. You need a double backslash in the %IF. The first backslash is escaped by the %IF, the second backslash needs to be there for %SEARCH. I updated above example. Try again.

-- PeterThoeny - 20 May 2008

Thank you, Peter! This worked as expected.

-- DevinBougie - 21 May 2008