For a long time I've been able to avoid too much manipulation of tables, but latterly I have had to do some work on them and I'm really disappointed with the current level of support. Let's quickly review the core support for tables, and the default plugins that use them:

• Core support: Virtually none. Tables are parsed by the renderer, but there are no specialised handlers to help plugins.
• SpreadSheetPlugin: parses tables from text. Builds bespoke non-object data structures to represent tables. Very complex code.
• TablePlugin: parses tables from text. Builds bespoke non-object data structures to represent tables. Code is extremely complex.
• EditTablePlugin: parses tables from text; has it's own rules for what constitutes a table. Builds bespoke non-object data structures to represent tables. Parser doesn't work in several edge cases. Code is extremely complex and fragile.

Maybe it's just me, but it seems glaringly obvious that all the above plugins can share the same data model. The advantages are obvious:

1. A single table parser can provide a consistent interpretation of what constitutes "a TML table". Support for table features such as ^ - which is currently isolated in TablePlugin - can be made more widely available.
2. Allow other text formats to leverage the same plugins - e.g. HTML::Parser can be used to parse an HTML table.
3. Simplify development of importers and exporters.
4. Required for the TopicObjectModel
5. In future, Plugin handlers can be defined - for example, for handling special table rendering - eliminating repeated re-parsing and inconsistent interpretations of TML.
6. Easily be serialised to JSON (or XML) for use by Javascript table editors
7. Easier to extend for multi-dimensional tables - protect the plugins from changes in TML

The main things that might throw a spanner in the works are:

1. Existing delicate assumptions in the plugin code make reverse compatibility very difficult (highly likely),
2. Some plugin user features are driven by the implementation, rather than usability; it may be difficult to deprecate these features.

At first glance, you might think that the HTML data model (column of rows) as used in the DOM might be appropriate. I'm not convinced of that, because of the poor way it handles row and column spans.

Other plugins that may benefit include:

• EditRowPlugin,
• RenderTableDataPlugin,
• EditTablerowPlugin,
• XpTrackerPlugin,
• WorkflowPlugin,
• FormQueryPlugin,
• ActionTrackerPlugin

Ideas? Suggestions? Code?

-- Contributors: CrawfordCurrie - 29 Apr 2007

Discussion

Outch, finger put into a delicate shortcoming. It would be so great to access table data easily. Infact, same holds for lists (bullet, enumeration, definition). They are neglected even more. Parsing TML lists is not as complicate as parsing TML tables but consists of enough gotchas to get it wrong. For example I started writing an RtfContrib module that translates TWikiMarkup to RTF and I don't want to rewrite a new TML parser or even copy-paste similar code from the core or some plugin into it as this is not sensible.

Crawford, you recently wrote the EditRowPlugin which also does its own table rendering. Is there any code in there to be leveraged as a core api?

-- MichaelDaum - 29 Apr 2007

Yes, but it doesn't handle row and col spans.

-- CrawfordCurrie - 29 Apr 2007

while you're thinking about tables - think about TopicObjectModel

like %INLCUDE{"Codev.FeaturesInProgress:table[features]"}%

and then the next step, to be able to select ranges of cells, or even.... several discontinuous ranges ....

-- SvenDowideit - 29 Apr 2007

the TopicObjectModel is point #4 on crawford's list

-- WillNorris - 29 Apr 2007

y, i wasn't quite clear - I think one of the easier ways to implement a table management system IS to implement the TopicObjectModel for tables. we've put doing it off mostly due to fear of the scope of the job - so splitting it by doing 'just' the table rendering system may be the kick start we need.

-- SvenDowideit - 29 Apr 2007

I wrote RenderTableDataPlugin to acccess table data to render it in different ways. The column sorting code is almost duplicate of TablePlugin.

-- ArthurClemens - 29 Apr 2007

OK, there seems to be some support for the idea. It may be necessary to go all the way to the TopicObjectModel to do this; let's bear it in mind; but I really don't want to go there because of the complexities of handling TWiki variables (which can generate new TML on the fly, and are what makes WYSIWYG so hard).

Here's what I did in EditRowPlugin

1. Parse tables out of the text of the topic and replaced them with placeholders (<!--N--> style)
   • In the full TopicObjectModel the table object would be an entry in the contents, so fully compatible there
2. Data structure based on the traditional HTML table model (TH, TD, TR, no THEAD, TBODY or TFOOT)
3. Used TWiki::Attrs to represent tag attributes
4. Wrote a serialiser that generates TML
5. Each table in the topic numbered from 1 upwards (this is to simplify passing table numbers in URL parameters)

On reflection that may have been wrong; I should have generated THEAD, TBODY, TFOOT tags for compliance with HTML 4, and to simplify the HTML parser. I didn't write any code to regenerate the implicit colspan in TML, because I wanted to edit the table, and an empty cell in TML is an implicit colspan.

What I have in mind is a simple table parser and API embedded in the core, and published through TWiki::Func. Without the TopicObjectModel this would be called repeatedly, whenever a subsystem wanted to parse tables out of topic content. The parser would not be subclassable, because if it were each plugin would end up subclassing it, to the confusion of all concerned. Instead, I suggest we implement C2:IteratorPattern, and possibly C2:ObserverPattern. I would prefer to implement C2:VisitorPattern, but I think it's too alien for most plugin authors to deal with.

-- CrawfordCurrie - 30 Apr 2007

Sounds good. I just want to pass over some TML containing one or more tables and get back an appropriate data structure to be further process?

I'd prefer to decouple the development of such a table parser in a TableParserContrib of its own and add it to the core later. That way we get code out earlier.

-- MichaelDaum - 30 Apr 2007

Good point; but I want to use it in the core (for table rendering)

-- CrawfordCurrie - 30 Apr 2007

What about putting the code in it's own module (TWiki::TML::Table?), and use a switch in the code to change from "traditional" (the current code) to "bleeding edge" (the code in that module)? Those that want to test, switch it on, those that want to run in production with MAIN switch it off?

If that's not possible... well, go for it and put it in the core.

XpTrackerPlugin will also benefit from such code, and I think that WorkflowPlugin and the TWikiForms code that read the form definition too.

-- RafaelAlvarez - 30 Apr 2007

On reflection, our experience with FuncUsersContrib suggests that we ought to do this as a contrib first, so we can iron out any problems before putting it onto the core. I really don't want to make it switchable in any other way. So, FuncTableManagementContrib here we come

-- CrawfordCurrie - 01 May 2007

... and I'd like to use such a contrib right away ... on 4.1.2

-- MichaelDaum - 01 May 2007

oh, and no pressure or anything :grin:

-- SvenDowideit - 01 May 2007