Query by Example searching

This idea started in CategorySearchForm. It would be useful to build a query-by-example form that does some AND/OR queries of form fields.

We could implement this with a new join switch in search. There is no change to the existing search if join is not defined. The join switch determines what parameters to combine in case you have a form where you want to AND combine the form fields into a search string.

Example: Assume we have a TWiki form with two fields, "Food" and "Drink". With the new AND search you can do a regex search like search="Food.*?Sushi;Drink.*?Sake". Now let's create a query by example form with two text fields, "Food" and "Drink". If you submit the form you get two parameters that need to be combined into the actual search string, so we have:

  <input type="text"   name="Food"  value="" />
  <input type="text"   name="Drink" value="" />
  <input type="hidden" name="join"  value="Food,Drink" />

-- PeterThoeny - 22 Mar 2002

A variant of the hidden "join" field syntax: How about defining the actual format of the merged search string instead of a simple "join" field. For example:

<input type="hidden" name="mergeformat" value="Food.*?$formfield(Food);Drink.*?$formfield(Drink)" />

The $formfield(name) would get expanded to the value of the field, e.g. the merged search would get "Food.*?Sushi;Drink.*?Sake". This is more flexible then a "join" but also more complex. Opinions?

-- PeterThoeny - 23 Mar 2002

I'm not sure I completely understand this. Does $formfield refer to the search form of the TWikiForms? Would it be better to have a syntax that doesn't assume knowledge of the METADATA format - i.e. the "Food.*?" part. What is the "?" for?

-- JohnTalintyre - 24 Mar 2002

$formfield(name) gets expanded to the value of an HTML form input field. The "?" in "Food.*?" is for a non greedy regex search.

Here is a concrete example: I would like to know all reported bugs on Solaris server. With the AND search I can accomplish this with this form:

Here I have an input field of type "text". This query is not user friendly because the user needs to be aware of (a) the possible values, (b) regular expression, (c) and to combine regexes with ";".

With the $formfield(name) enhancement I could build a form like:

This query will work once the mergeformat parameter is implemented. The form has a drop down named "classification" and a text field named "serveros". The merge format defines how to combine those two form fields into a search string:

<input type="hidden" name="mergeformat" value="Server OS\:.*$formfield(serveros);META.*TopicClassification.*$formfield(classification)" />

The resulting search string will looks like the one in the first form; it will be passed as the search string to TWiki::Search.

This new mergeformat parameter is more complex but allows you to build query by example forms that pull data out of unstructured topic text and TWiki forms (if needed).

A QueryTWikiFormPlugin could be created to hide this complexity, it could build an HTML form of all (or a subset of) the TWiki form fields. This is related to ExtendingFormControls.

-- PeterThoeny - 24 Mar 2002

Related to this is EditControlsBasedOnFormDefinition: When you build a query-by-example form you do not want to manually update the form controls when the form definition changes.

-- PeterThoeny - 12 Jul 2002

From the name of the FormQueryPlugin, one might expect the FQP to address exactly this. Not so... but I think having this feature is a key priority in getting significant new functionality on TWiki.

-- MartinCleaver - 01 Oct 2004