Feature Proposal: Make MetaDotPm more extensible

Motivation

Meta.pm can store keyed and unkeyed values. It uses keys for attachments and fields. The key is hardwired for these to be the string "name" in the subroutine _keys() .

Any application that wants to add a new kind of meta data is required to update Meta.pm and extend the definition in that subroutine. In particular, that makes it impossible to write a plugin storing information in metadata without modifying MetaDotPm. (Yes, I realize access to metadata is not encouraged in FuncDotPm, but it is still possible.)

Description

Change Meta.pm to always require the key "name" for keyed access.

The flexibility afforded by having other keys defined in key() is not used in TWiki and seems hardly worth the price of not being able to add other keyed metadata without changing the code.

-- ThomasWeigert - 19 Feb 2005

Impact and Available Solutions

Note: Patch is attached as https://twiki.org/p/pub/Codev/MakeMetaDotPmMoreExtensible/meta_with_key_name.diff. The patch is against the cairo release.

Documentation

There is no real documentation with MetaDotPm, so it is difficult to give the changes here.

In short,

• All keyed metadata (i.e., metadata where one can have more than one entry for the same type) are keyed by name
• Keyed metadata is inserted with the method TWiki::Meta::putKeyed, rather than TWiki::Meta::put.

Examples

Formulate SimpleTableEntryUsingForms as plugin.

Implementation

As discussed below, the implemented solution uses name as the sole key for keyed metadata (this is consistent with current TWiki usage). Consequentially, applications can add new types of metadata without changing TWiki core.

In SVN 4272.

---

Discussion:

Good idea. The only reason I never bothered doing it before is that I don't really like the implementation in MetaDotPm, with it's restricted two-level view of the world. I think we need to ditch it and move to a proper TopicObjectModel. In that model, we would have a generic storage object (call it a "Map") and a TWiki::Topic would be a subclass. Meta could either be a subclass as well (i.e. the Topic object would have a "meta" entry) or it could be merged with the topic object.

I'm nervous that if we make Meta too useable in the current implementation, we will have to deal with horrible compatibility problems in the future with all the plugins that jump on the chance to manipulate it.

-- CrawfordCurrie - 20 Feb 2005

Crawford, you make two separate points here, which I'll address separately....

First, regarding a better implementation of MetaDotPm... I agree that this should be the long-range plan. Well, actually, the real plan should be to make the storage pluggable in that one could substitute a data base for the file system, for at least some aspects of the topic data. But, in the short term, I am suggesting something much more mundane:

• Remove the unmotivated restriction in MetaDotPm that the only keyed metadata are attachments and form fields.

This can be done in two ways: either use the same key (e.g., "name", for all keyed metadata) or allow the keys to be extended flexibly. The latter seems to provide flexibility that serves little purpose and comes at a cost in execution time.

Secondly, you rightfully raise a concern on plugins stomping on top of each other. Note that, unfortunately, this concern is much wider than metadata: Plugins can already conflict with both their name (need not be the file name) as well as the tags they define. Two separate plugins may define a new TWikiVariables, say %CONFLICT%, and obviously, only one will be invoked unless they have different arguments.

Here are some ideas to mitigate:

• To avoid metadata conflicts, we may give Plugins separate namespaces in that an API would enforce, e.g., prefixing the name of the metadata with the plugin name.
• This approach seems not workable for tags, as users would be seeing these prefixes in their text. At a minimum, we could define a mandatory handler that states the tags a plugin uses. A script (similar to testenv) could then invoke those handlers and detect conflicts.
• We could keep a registry of tags and metadata names on twiki.org that Plugin authors should check when defining their tags.

-- ThomasWeigert - 20 Feb 2005

http://arstechnica.com/reviews/os/metadata.ars/1

-- WillNorris - 29 Apr 2005

This is a sensible small enhancement.

The namespace issue with Plugin introduced meta tags can be done by convention (e.g. a GooeyPlugin would introduce META:GOOEY and META:GOOEY:MESS). The Plugin API could also enforce that.

-- PeterThoeny - 01 May 2005

Done.

-- ThomasWeigert - 06 May 2005