Feature Proposal: Easy url query handling

Proposal for a way to set and change url params / query string.

Motivation

TWiki lacks the ability to manipulate the query string. We have QUERYSTRING to get the complete string, and URLPARAM to get individual parameters, but to set parameters we have to resort to repeated, error-prone spaghetti code.

For instance to create 5 sort options we have to write 5 times a line like this:

%SCRIPTURL{"view"}%/%BASEWEB%/%BASETOPIC%?category=%URLPARAM{category}%;limit=%URLPARAM{"limit"}%;reverse=off;maxcount=%URLPARAM{maxcount}%;sort=formfield(Title)

while here the only changing element is the sort=formfield parameter - all the rest is retrieved from the current query string.

Things would be easier like this:

%SCRIPTURL{"view"}%/%BASEWEB%/%BASETOPIC%?%QUERYSTRING{keys="sort" values="formfield(Title)"}%
%SCRIPTURL{"view"}%/%BASEWEB%/%BASETOPIC%?%QUERYSTRING{keys="sort" values="formfield(Author)"}%

To set multiple values at once:

%SCRIPTURL{"view"}%/%BASEWEB%/%BASETOPIC%?%QUERYSTRING{keys="sort,limit" values="formfield(Author),25"}%

Description and Documentation

QUERYSTRING would take all existing values (if any) and set or change values in the keys and values parameters. The generated html would look the same as the current way.

Everything would be really condensed if Wikiword and bracket notation would allow a query string. The above example would be as short as:

%BASETOPIC%?%QUERYSTRING{keys="sort" values="formfield(Title)"}%

ArthurClemens

Impact and Available Solutions

Implementation

Discussion

Looks useful. I think it is more intuitive to simply allow name="value" parameters, such as %QUERYSTRING{ sort="formfield(Author)" limit="25" }%. Possibly add a switch to turn off (on?) the existing params, such as _noexistingparams="on".

-- PeterThoeny - 13 Dec 2006

When I looked at extending QUERYSTRING before, I gave up, because it was so constrained in it's specification. Basically QUERYSTRING is nothing more than an interface to CGI::query_string and as such inherits all the constraints of encoding etc. So I added QUERYPARAMS, which already supports formatting of the query parameters using format and separator. The biggest problem with it at the moment is the poor support for encoding; it requires embedding of ENCODE to format the parameters correctly, which is clunky.

However, I think the whole approach is a messy hack and is not really right for what you want to do. As you know, HTTP has two ways of obtaining data from a server, via GET and PUT requests. The QUERYSTRING is the string that would be used in the URL if the URL was a GET (even if it is a POST) and is necessarily incomplete (it will be missing form components such as multipart-encoded) and dangerous (a POST request can result in a QUERYSTRING that is extremely long e.g. an edit gets the entire topic content).

HTML has two ways of passing parameters to a URL. You can encode them in the URL, which is what you are trying to do here, or it can encode them using a <form> with one or more submit buttons. For full safety, you should always use the form approach, thus:

<form action="%SCRIPTURL{"view"}%/%BASEWEB%/%BASETOPIC%">
%QUERYPARAMS{format="<input type=hidden name='$name' value='%ENCODE{$value}%'/>"}%
<input type="hidden" name="anextraparam" value="extravalue" />
<input type="submit" value="Do your worst"/>
</form>

Trying to manually build URLs with the query string is difficult and clunky, and if you find a GET URL blows up and have to convert to a form, it's a real PITA. So what I personally think we really need is a way for a single syntax to be able to generate either a form for a post, or a URL, with an optional passthrough. For example:

%URI_REFERENCE{url="%SCRIPTURL{view}/%WEB%/%TOPIC%#blah" passthrough="on" anextraparam="extravalue" style="form" method="post"}%

By switching from style="form" to style="uri" it can generate a URI instead of a form.

While you are thinking about this you might also consider another approach specific to TWiki URIs, thus:

%SCRIPTURL{"view" web="%WEB" topic="%TOPIC%" passthrough="on" anextraparam="extravalue"}%

where web and topic default to the current web/topic.

-- CrawfordCurrie - 14 Dec 2006

Some apps might need to manipulate the parameters on the full URL, on the URL path, or only on the query string. To be consistent it can be added to all three: VarSCRIPTURL, VarSCRIPTURLPATH, VarQUERYSTRING. For example, the printable link can use a SCRIPTURL with modified query string to easily link to a printable view, including all existing parameters.

Suggested params:

• %SCRIPTURL{ "view" web="%WEB%" topic="%TOPIC%" querystring="on" extraparam1="value1" extraparam2="value2" }%
• %SCRIPTURLPATH{ "view" web="%WEB%" topic="%TOPIC%" querystring="on" extraparam1="value1" extraparam2="value2" }%
• %QUERYSTRING{ suppressquerystring="on" extraparam1="value1" extraparam2="value2" }%

The querystring="on" adds the query string, name is consistent with QUERYSTRING variable. The extra parameters replace existing parameters of same name, if any.

Consider also adding a encode="entity" and encode="url" param, orthogonal to VarURLPARAM.

-- PeterThoeny - 15 Dec 2006

Hmmm. I do not like to add querystring="on", especially when querystring="on" is the only possible interpretation of the extra parameters. Also, why would you want encode? This is a URL, as indicated by the name SCRIPTURL, so url encoding is the only encoding that makes sense, and it should always be applied. Can you give a use case for encode="entity"?

-- CrawfordCurrie - 16 Dec 2006

It could be something like newquery="on" to not copy the existing query params.

-- ArthurClemens - 16 Dec 2006

No, it couldn't be, because that is not the way SCRIPTURL works at the moment. I really can't understand the objection to passthrough="on" - it is by far the most user-friendly approach. The fact that I misinterpreted what querystring="on" was meant to mean is a sure indication of that.

-- CrawfordCurrie - 17 Dec 2006

The idea of passthrough is ok, but the name does not convey what is passed.

Would URI_REFERENCE create the complete form, or just one <input> line? If the complete form, how to create multiple <input> lines?

-- ArthurClemens - 17 Dec 2006

It seems all agrees on the proposal but as usually not the spec.

Can we get this spec discussion going again or do I simply park the proposal as abandonned.

Is Arthur the driver if a spec is agreed?

-- KennethLavrsen - 08 Apr 2007

Parking the feature until driver shows up to drive the spec and implementation.

-- KennethLavrsen - 23 Apr 2007

Signing up for the driver.

-- ArthurClemens - 23 Apr 2007

Thanks Arthur!

I am not clear on the spec, could you update the spec?

-- PeterThoeny - 24 Apr 2007

I am parking this.

-- KennethLavrsen - 03 Jun 2007