Feature Proposal: content of SEARCH{...} format tokens ($text, etc.) to be able to be encoded

Motivation

You may need to encode $text content, not the entire format parameter content. For example, when you have format="| something | $text |", if encode="entity" is present, not only $text content, but the entire format content is subject to the encoding, which means it cannot be rendered as a table.

This is also the case with $formfield(...), $query(...), and $pattern(...).

Description and Documentation

Introduce encode:ENCODING parameter to aforementioned format tokens. For example:

• $text(encode:entity)
• $formfield(Summary, encode:safe)
• $query(text, encode:quote)
• $pattern(.*?(from here.*?to here).*, encode:url)

Examples

Impact

Implementation

-- Contributors: Hideyo Imazu - 2013-01-31

Discussion

Good point. Looking at FormattedSearch, the same can be said for $summary and $formfield().

Instead of a new parameter, how about adding new tokens to $text, $summary and $formfield()? For example, $text(encode:entity) would entity-encode the the text, and $formfield(Summary, encode:html) would html-encode the form field called Summary. This is consistent with existing tokens of $summary and $formfield().

-- Peter Thoeny - 2013-01-31

Makes a lot of sense. I've updated the proposal accordingly and matched the topic name.

-- Hideyo Imazu - 2013-01-31

It is logical to add the encode option also to $pattern(). It might be tricky to parse that reliably. Currently the end of the pattern is defined by a .*), e.g. as documented the pattern must end in .*. If we allow optional encode, we need to tweak the regex to reliably detect it and still allow arbitrary patterns. Untested:

$out =~ s/\$pattern\((.*?\s*\.\*)(, encode\:[a-z])?\)/getTextPattern( $text, $1, $2 )/ges;

-- Peter Thoeny - 2013-01-31

Feature accepted by JerusalemReleaseMeeting2013x02x01.

-- Peter Thoeny - 2013-02-01

While implementing the enhancement, I realized that $summary doesn't need encoding because it's table safe. And $query(...) should have encoding option.

-- Hideyo Imazu - 2013-02-08

Not sure, there might be a need to URL-encode summary or query text.

-- Peter Thoeny - 2013-02-08

I have not problem providing encode:type option to $summary if needed. But as far as I see, it's cleaned in various way already. At least the value doesn't contain new line or the vertical bar character. So I cannot see a use case.

To put a text area field of a TWIki Form $query(Fieldname) can be used and to put it on a table, it needs to be encoded. In this particular case, $formfield(Fieldname) does the job, but I guess there are cases where $query(...) having no synonym in $formfield(...) yield new line and/or vertical bar containing results, which needs to be encoded in some cases.

-- Hideyo Imazu - 2013-02-09