Feature Proposal: Refactor format rendering to use ResultSets (adds paging, consistency and speed)

Motivation

We've known for a long time that we need to add scalable paging to our Searches, GROUPs and such. I propse we do this, by extracting the format= code from all the TML, and move that code to take a TWiki Iterator (the basis of what will become ResultSets). (and we expose that via Func). That way, we create one consistent formatting system, and will be able to add paging in a generic way to all tags.

AddUSERSLISTandGROUPSLIST is another example where we need to implement Paging - the Joomla and LDAP use cases are known to have way more users than should be displayed in one hit.

Description and Documentation

Extract the formatting stage from SEARCH and other tags, re-write it to use TWikiIterators, with filtering, and late evaluation of Iterators where possible.

Crawford and I have been working towards this over the last years, but it needs a final push (ok, quite a lot of work) to get it completed.

Include adding and merging the TOM specifiers so that we have the same syntax in If, query SEARCH and format strings. see AddAttachmentsVarToFormattedSearch

probable outcomes

• speedup oportunities - via late evaluation of iterators
• consistent paging and formating parameters and code that will be reuseable by plugins
• Would enable ResultSets that could then be used for nested operations
• =%FORMAT{}% TML func that can be used to apply a format to any list of results - whether that is a SEARCH result, GROUP list, Saved Query Result - basically informs TWiki howto render ResultSets.

Examples

Impact

Implementation

-- Contributors: SvenDowideit - 15 Dec 2007

Discussion

Don't forget to add the DateOfCommitment I added 2007-12-15.

I support this proposal. It is a very much needed feature. Both for the core and for plugins.

-- KennethLavrsen - 17 Dec 2007

What does "using TWikiIterators" mean for the end user who's used to type format="..." ? I don't know much about the concept of iterators so any clarification would be appreciated

-- CarloSchulz - 17 Dec 2007

for the user, the only thing they might notice, is that all the format= statements use a common syntax. Right now, some TWikiVariables don't support $n some use $date - that sort of thing.

mostly, no change except for consistency, and a common paging scheme for all mulitple result tags.

-- SvenDowideit - 17 Dec 2007

An iterator would also be useful in templates. We could do away with SPLIT, and simplify code for lists of buttons.

For iterating over search results, see the proposal in SearchResultsPagination. Note the sort step and the need to have iterating controls.

-- ArthurClemens - 17 Dec 2007

Aha, what Arthur mentions brings up yet another proposal that will be more noticable... ResultSets - which will thus give us the need to add a =%FORMAT{}% function, that will use a specified (or named) result (aka, iterator, setting, search, GROUP)

-- SvenDowideit - 21 Dec 2007

Seems all are happy. Accepted by 14 day rule.

-- KennethLavrsen - 02 Jan 2008

another point - could do with the raw='on' working everywhere - essentially it would retreive and render the format, and then would prevent twiki from rendering it further.

-- SvenDowideit - 01 Jun 2008

I don't understand that. raw=on means "use a specific template for this page" to the view script. I had been thinking that TinyMCEPlugin should respect "raw=on" as synonymous with nowysiwyg=1. But you seem to be suggesting something else...

-- CrawfordCurrie - 01 Jun 2008

no, i think most users see ran=on means show me the unrendered version. so TML{format="$web.$topic" raw=on} would render something like Web.TopicName

-- SvenDowideit - 01 Jun 2008

This is essential to ideas like RelaxRegisterTagHandlerCallingContext

-- SvenDowideit - 05 Sep 2008

Useful feature proposal, but: I am setting this to parked and no committed developer. Please feel free to flip that and own & implement.

-- PeterThoeny - 2010-08-01