While the configuration is a lot simpler now with configure there are still a few problems:

1. there are two configuration files (LocalLib.cfg and LocalSite.cfg)
2. LocalSite.cfg is in lib, which should be read-only.
3. Because of the mess of paths in setlib.cfg, command-line scripts have to cd to bin and require setlib.cfg to work
4. It is quite hard to patch LocalSite.cfg

There are two main problems here:

1. Local @INC setup
2. Including the TWiki configuration

Local @INC setup

I think we need to take a radical step, and dump setlib.cfg/LocalLib.cfg completely. This requires the installer to add two paths to @INC: lib (twikiLibPath) and lib/CPAN/lib= (localPerlibPath). This can be done easily in the apache configuration:

SetEnv TWIKILIB /usr/local/twiki/lib:/usr/local/twiki/lib/CPAN/lib

and then

BEGIN { unshift(@INC, split(':',$ENV{TWIKILIB} || '')); };

in each of the scripts, replacing the current load of setlib.cfg. This puts the paths much more "in your face" for the installer, which IMHO is a Very Good Thing. It's all very well hiding stuff for the simple configs, but it creates a nightmare for anyone who isn't simple.

For command-line scripts, -I is recommended for adding to the @INC path.

Including the TWiki configuration

The first thing is the location; lib is the wrong place. I propose we adopt a new user-writable directory conf and move LocalSite.cfg there. TWiki.cfg should remain where it is; it is not intended to be writable.

Currently configure works by "parsing" TWiki.cfg and LocalSite.cfg to extract special marker comments that are used to determine display types for data. This is OK insofar as it goes, but there are a couple of problems with it:

1. It is hard for an extension to add to a select declared in TWiki.cfg. For example, if you have
   # **SELECT male,female**
$cfg{Sex} = 'unknown';=
    then adding "neuter" to that set because an extension supports it is not supported at the moment. It should be hard to do; for example, in =!LocalSIte.cfg the extensions installer script could add:
   =# **SELECT_MORE {Sex} neuter**=
2. Collections have poor support. At the moment only hashes are sort-of supported, just be virtue of the fact that you can write $cfg{Drugs}{Grass}{Toke} - but that doesn't help if you need to be able to add $cfg{Drugs}{Coke}{Snort}. Some sort of mechanism for specifying collection types is required.
3. It would be cool if we could use the same configuration specification in the installers, so that the installer for an extension can prompt for an initial configuration. I had this problem with the MailInContrib, which prompts for initial mailbox setup (which is, interestingly, also a collection; an array of hashes.

-- Contributors: CrawfordCurrie

Discussion

-- CrawfordCurrie - 23 Feb 2006

• Setenv TWIKILIB can be also be put in .htaccess, so it is a very good solution as it can be used on hosted sites where you do not have access to the central httpd.conf
• (very minor) as a unix-head, having the conf/ dir named etc/ would make me warm inside
• Maybe I do not understand the problem, but maybe just have a file per config item would simplify things: e.g., instead of $cfg{sex} have its value in the file conf/sex.conf. Then LocalSite.cfg would not be editable anymore, it would be generated by a cat *conf >LocalSite.cfg

-- ColasNahaboo - 23 Feb 2006

So, instead of having a single, monolithic configuration file (LocalSite.cfg), there would be several files, one for each "extension" and one for the Core, and configure can be made smart enought to be able to edit them all.

Sounds nice. Don't Apache 2.0 and xinid use that same mechanism?

-- RafaelAlvarez - 23 Feb 2006

I don't think a file per config would simplify things, unfortunately. When you had a configuration item in your hand, you would have to decide where to put it. A single flat file makes that easy.

Perhaps you could expand on the idea (preferably with code?)

-- CrawfordCurrie - 23 Feb 2006

can we have a TWIKILIB and a TWIKICFG ? on a system wide install (where TWIKILIB is in /usr/lib/perl5), i would like the system wide cfg to be in /etc/twiki, but if a user wants their own twiki, all they need is the bin scripts, and thier own TWIKICFG from htaccess.

next of couse i'd like a similar mechanism for templates, locale, and pub and data - where there are system wide topics (ie the diestributed ones)

in fact we could take this further - if I install TWiki into the system, I should be able to cerate a pseronal TWiki, using just a TWIKICFG setting and a re-writeurl to point to the system's bin files

Neat!

now, remind me again why TWIKILIB needs to be in .htaccess? surely that can be in the LocalSite.cfg file, all the bin script needs is how to get to it?

we do need to provide a validation od library versioning and dependancies though - I've had one or two hickups when I've install the debian TWiki (into /usr/lib/perl5), where my development twiki uses the old cairo libs without me realising.

-- SvenDowideit - 23 Feb 2006

For the TWiki libraries, have a look at CPAN:perl5lib which implements in two lines of code how to respect the environment variable PERL5LIB in taint mode (all env vars are tainted!). In my opinion it would make sense to use PERL5LIB instead of TWIKILIB, and perhaps just copy the two lines of code from perl5lib.pm to TWiki's binaries if you want to avoid another CPAN dependency. OTOH, if we copy the lines, we can stick with TWIKILIB as well - and possibly handle TWIKICFG in the same way.

-- HaraldJoerg - 24 Feb 2006

Moving LocalSite.cfg out of lib is sensible, this also allows multiple TWiki installations on one server that point to the same TWiki lib.

Related, a transparent handling of Plugin configurations would be helpful. That is, the Plugin package would contain a configuration file (descriptive text and default settings) that gets picked up automatically by configure after installing a Plugin. One example is sensitive data that should not go into Plugin "Set" variables, such as file path to executables, or file path to output directories.

As always, keep it simple and fast. Better not to use a CPAN lib just for this.

-- PeterThoeny - 27 Feb 2006

The BuildContrib includes the functionality (and has done for some time) that allows POSTINSTALL scripts in plugins to modifiy LocalSite.cfg. The reason I did this rather than supporting multiple config files was for 4 reasons:

1. Files shipped with plugins should be read only, otherwise they get overwritten on upgrade
2. Individual files would require discovery, increasing the code burden.
3. I want to use the configure interface to configure plugins.

-- CrawfordCurrie - 27 Feb 2006

I was thinking that "external" config files, like plugin and contrib config, should go in a "third" file (Extensions.cfg?) that is also readed by configure, and modifications to the defaults there written to LocalSite.cfg. So, this mean that there would be 3 files:

• TWiki.cfg for Core configuration
• Extensions.cfg for extensions default configuration (plugins & contrib)
• LocalSite.cfg for "local" overriding the other two files.

The POSTINSTALL scripts can be made to modify Extensions.cfg instead of LocalSite.cfg. Or Extensions.cfg can be "built" by configure by discovering configurations files distributed in a known directory (conf?). The first option has the advantage that there is no need for "staleness" detection.

-- RafaelAlvarez - 27 Feb 2006

Would it be sensible for a newly written plugin to store its configuration data in the conf directory, even if it is designed to run under TWikiRelease04x00? It would need its own bin/foopluginconfigure for 4.0, but then this could be integrated as a sub into Edinburgh's bin/configure without much extra effort.

-- HaraldJoerg - 01 Apr 2006