Module name TWiki::Prefs.pm PrefsDotPm Location TWIKIROOT/lib/TWiki/Prefs.pm Summary TWiki preferences handling on site-level, web-level and user-level Primary Author PeterThoeny CVS history CVS:lib/TWiki/Prefs.pm CVS alpha CVSget:lib/TWiki/Prefs.pm Contributing authors (see CVS History) Is Class NO First Release to be filled out

File Hierarchy   TWIKIROOT   lib   TWiki   Prefs.pm

Purpose

This module reads TWiki preferences of site-level, web-level and user-level topics and implements routines to access those preferences

Used by

This module is primarily used by ...

Please see CodevDocumentationProject and CodevDocumentationProjectDev for comments on the format of these pages.

Note:	Below documentation is extracted from the currently installed TWiki::Prefs Perl module, which is done by the PerlDocPlugin

package TWiki::Prefs

The Prefs class is a singleton that implements management of preferences. It uses a stack of TWiki::Prefs::PrefsCache objects to store the preferences for global, web, user and topic contexts, and provides the means to look up preferences in these.

Preferences from different places stack on top of each other, so there are global preferences, then site, then web (and subweb and subsubweb), then topic, included topic and so on. Each level of the stack is tagged with a type identifier.

The module also maintains a separate of the preferences found in every topic and web it reads. This supports the lookup of preferences for webs and topics that are not on the stack, and must not be chained in (you can't allow a user to override protections from their home topic!)

ClassMethod new( $session [, $cache] )

Creates a new Prefs object. If $cache is defined, it will be pushed onto the stack.

=begin twiki

ObjectMethod finish()

Break circular references.

ObjectMethod pushPreferences( $web, $topic, $type, $prefix )

• $web - web to read from
• $topic - topic to read
• $type - DEFAULT, SITE, USER, SESSION, WEB, TOPIC or PLUGIN
• $prefix - key prefix for all preferences (used for plugins)

Reads preferences from the given topic, and pushes them onto the preferences stack.

ObjectMethod pushWebPreferences( $web )

Pushes web preferences. Web preferences for a particular web depend on the preferences of all containing webs.

ObjectMethod pushGlobalPreferences()

Add global preferences to this preferences stack.

ObjectMethod pushPreferencesValues( $type, \%values )

Push a new preference level using type and values given

ObjectMethod mark()

Return a marker representing the current top of the preferences stack. Used to remember the stack when new web and topic preferences are pushed during a topic include.

ObjectMethod restore( $mark )

Resets the preferences stack to the given mark, to recover after a topic include.

ObjectMethod getPreferencesValue( $key ) -> $value

• =$key - key to look up

Returns the value of the preference $key, or undef.

Looks up local preferences when the level topic is the same as the current web,topic in the session.

ObjectMethod isFinalised( $key )

Return true if $key is finalised somewhere in the prefs stack

ObjectMethod getTopicPreferencesValue( $key, $web, $topic ) -> $value

Recover a preferences value that is defined in a specific topic. Does not recover web, user or global settings.

Intended for use in protections mechanisms, where the order doesn't match the prefs stack.

getTextPreferencesValue( $key, $text, $meta, $web, $topic ) -> $value

Get a preference value from the settings in the text (and/or optional $meta). The values read are not cached.

ObjectMethod getWebPreferencesValue( $key, $web ) -> $value

Recover a preferences value that is defined in the webhome topic of a specific web.. Does not recover user or global settings, but does recover settings from containing webs.

Intended for use in protections mechanisms, where the order doesn't match the prefs stack.

setPreferencesValue($name, $val)

Set a preferences value. The preference is set in the context at the top of the preference stack, whatever the current state may be.

The preference is not serialised.

ObjectMethod stringify() -> $text

Generate a TML-formatted version of the current preferences

Contributors:
-- MartinCleaver - 23 Jun 2002
-- PeterThoeny - 02 Feb 2004

Discussions

In MetaDataSucksMeasurement I presented statistics that demonstrated a major performace hog was the handling of preferences, and elsewhere that seperating out the rpferences to do with acess control would allow some decisions in, for example, bin/view, to be maed earlier and so save processing.

I'm working on a "database" solution for preferences and have found some problems with the deisgn of lib/TWiki/Prefs.pm. Basically its awful.

Why is it awful? Primarily it is designed in such a way that any changes to it have to be sweeping. For example:

• The subroutine "getPrefsFromTopic" doesn't do what its name says.
  • It merges the prefs from the topic into a global array
  • Ths relies on "side effect", which is acknowledged as being unstructured - back to the spahgetti-code days!
  • It prevents caching of the preferences of any topic.

What I'd like to see is a primitive, later an object based set-up where the cach is on "class data" rather than "instance data", whereby

  (@keys, @values, @finals) = getTopicPrefs( $web, $topic, $prefix) ;

Those results can bow be cached.

The curent functionality of getPrefsFromTopic() would call a routine that

• looked up the cache for that web/topic combinaton
  • if not found then getTopicPrefs() and add to cache
• merged the values

The second thing I don't like about getPrefsFromTopic() is that it also get values from the META From Fields where the value has an upper case "S in it:

• Sucess but not sucess
• sucesS but not sucess
• traSh but not trash
• SSSuperb but not sssuperb

I'm sure there is a reason for this but I can't find one.
Regardless, it seems to be there merely becuase getPrefsFromTopic() has to make use of TWiki::Store::readTopic(), which has the side effect of processing the the META data, because META data, like access control, is in-band.

(I'm sure someone is going to point out that moving metadata out of band may better structure the data storage, better structure the code and lead to an OO model, but will break some things like %SEARCH% relying on the META data being in-band. However, its already broken-by-design (qv) since a META statement edited in by the user looks just like a system generated META. See AccidentalMetaDeclInTopicTextIsNasty)

Ultimately this has to head into OO, not just for the basic design but to allow "overrides" for custom plug-in alternative methods. I've been experimenting with different kinds of databases (i.e. external store) and tools for conversion. My preferences may not be yours ....

-- AntonAylward - 15 May 2003

Anton, I wholeheartedly agree with you. For too long have I read things like "we can't put the topic subscribers in the meta data because the destination topic might be locked by someone editing the content of the topic." (Horribly, this design decision is affecting ImmediateWebNotifyPerTopic). MetaData operations are by nature quick and atomic and should be independent of content editing processes.

If you can code your suggestion I will back you in getting it adopted into the core. I also think you should nominate yourself (and others) in CoreTeamNominationDiscussions.

-- MartinCleaver - 16 May 2003

... and so shoould be access control oeprations ...

Thank you for your sentiment. I don't think I'm core team material; but the standards of the core team I'm not just a hetrodox, I'm a raving, revolutionary heretic. The trouble is that I accept the same basic premis - that perl and wiki are worth doing.

I'm not really a coder, per se, but an ace because I work through building the spec and then implementing from it. Little "production" happens except experimentation before I know fully and have fully documented what I'm going to do. I discuss and thrash out ideas endlessly and make measurements and trials and notes. Its like putting up a building; you need the plan up front, not a 'make it up as you go along'.

As you've observed, TWiki is stable - alpha is bugfixes and even many of the innovations being discussed aren't being merged in to the core. Is this protectionism or .... inertia ... or what? These are good people, but like the priests of established religions they have an investment in what is, not what might be.

Part of the problem is that while from an idealistic POV I beleive in incrementalism, there is too much of the core of TWiki that is not orthogonal it is too coupled. The code in Meta.pm is about the only thing that's "clean". Most eveyhting else was grouped by original function then grew. Strict boundaries for Modules, principles, partitioning of control, presentation and function, are all missing.

The prime example of this is that the access control, metadata and body are all in the same file is the best example of this. You can't read any one without reading the other. I know of no storage system like that. Think of UNIX; you can perform a stat(2) to find out the access permissions for a file without opening the file and inspecting its contents. Under VMS, the type of the file - block, fixed length records, length prefixed records, etc (14 categories if I recall) - which is the closests I can think of to metadata, could be accessed without actually reading the file.

Despite there bing a module called =Access.pm=, the access control is not localised there, nor is it arranged in such a way that its functions can be inherited and over-ridden by an "experimental" new method of some parts. Plug-ins don't work by being @ISA Plugin and running their rendering drivers then their SUPER class driver.

... and trying to optimize by using load on demand - things like AUTOLOAD - is a nightmare given how teh code is structured.

For example, the implementation of Store.pm is such that these things cannot be easily decoupled. The change cannot be made in small, evolutionarly steps. It has to be large steps and it can't simply be a Darwinian mode, there has to be an over-riding direction and objective. An ideal design would allow the methods to be over-riden, but that assumes a coherent and othogonal base class design.

__And yes, I'm working on it, and yes, I'm writing it up as I work it out and yes, it will appear here one day "real soon now"__. In the mean time I've got a life.

And yes, I'm soliciting help. I don't have the charisma or arrogance to think I can undertake this on my own. And yes, having the core team say "Well like Fred Brooks said, we built that to learn and throw away, not lets do it for real" would be great.

-- AntonAylward - 16 May 2003

Despite the several ideas of optimizing preferences handling in TWiki, I was wondering why PrefsDotPm does not use a hash but two lists for keys and values instead. There is a note in the source telling "Do not use a hash, because order is significant", but from the code I see no reason why this should be necessary. When accessing (reading) the values, the code iterates through the list and uses the first match; from the way new prefs are added, there is no second occurence in the list at all. When adding the key/value is dropped if FINALPREFERENCES tells so, or a first match (if any) is overwritten, otherwise appended to the list. Special case FINALPREFERENCES, where new values are appended.

So using a hash here would probably be faster and does not break anything. And, it is a better starting point for optimizing.

-- MichaelRausch - 27 May 2003

Just to give you an update: after cleaning up the key/value lists vs. hash (See above) a little bit, I turned to implementing a cache for preferences using the Storable and the Memoize module. A first shot (this afternoon´s work) gives the following figures on a test page:

mod_perl	what	time to load
no	unmodified source	2.7 sec
no	using hash instead of lists	2.5 sec
no	storing preferences	2.4 sec
yes	unmodified source	0.8 sec
yes	using hash instead of lists	0.6 sec
yes	storing preferences	0.5 sec

Especially when using mod_perl, the gains are considerable (~40%). Right now, I´m using a hash of hashs for preferences per web (that is site + web preferences, was %altprefs before) and a hash of hashs per web+user (that is site + web + user preferences, was %prefs before). What is still to be done is refreshing the cache in case the preferences change, I´ll implement that via timestamps.

Code will be published as soon as possible (next few days)...

-- MichaelRausch - 28 May 2003

Great stuff! I've been playing with converting the settings to an external database and using a Tie. I'd played with botht he hash and the array and made the same observation.

What you should now do is modify the code so that it builds a has on a per topic basis and caches it. The routine that builds the glob of settings - Site, Web, User - can make multiple calls on that.

My analysis showed that in accesing some pages such as WebHome with the big table, there were a large number of calls to build settings lists. A cache would produce significant run-time improvement becuase of the reduced IO, as well as the reduced processing involved in parsing that Michael's table shows.

And of course migrating the settings out of band would achieve a few more enhancements:

1. It would produce even more performace improvement!
2. It would begin to decouple the storage of settings
3. It would produce code fragements that are more easily adopted to a Object Oriented revision of Twiki.
   (That's where my efforts are going these days, I can't manage all the code by myself!)

-- AntonAylward - 28 May 2003

New, optimized version attached to DatabaseForPreferences ... but still needs some polishing.

-- MichaelRausch - 04 Jun 2003

A question was asked above about why search for "S" in form attributes. Each attribute is represented by a single letter, "S" stands for setting, any order allowed.

See UsingFormsForSettings which discusses this change.

-- JohnTalintyre - 04 Jun 2003

Another question from above. I believe order is important as some setting can't be overridden. I'm sure Peter has answered this question somewhere in Codev. Prefs can be a bit confusing, but it does a lot with a little code - there a lot to be said for this.

-- JohnTalintyre - 04 Jun 2003

Has anyone written a routine to dump out all preference values? Thanks.

-- MartinCleaver - 12 Dec 2003

here you go Martin - it doesn't do WebPrefernces or Session preferences (thats for later..)

• dumpPrefs.diff: add global TWiki preferences dumping to testenv

and you get a TWIKIDISTRIBUTIONVERSION value for free - see TWikiDistributionVariable

should we add this to core?

-- SvenDowideit - 02 Jan 2004