Proposed: Namespace Control

Summary:

This proposal adds a single setting to WebPreferences, NAMESPACEWEB. This setting controls if the topics in this web are included in a larger named namespace.

Background:

This proposal addresses a need for additional OrganizingPrinciples in addition to the concept of Webs, which is a key organizing principle used in TWiki. After prolonged discussion, it appears that control of the scope of topic names is sufficient to provide the basis for other OrganizingPrinciples without adding undue complexity. Also, this proposal provide backward compatibility such that topics can still be discovered by using the full webname-prefixed topic name.

To help in the description of these controls, the following definitions are suggested:

• TWikiWeb: the customary container for topics provided by TWiki today. Each provides a number of WebPreference settings, notifications, etc., and are physically located in a separate directory.
• NameSpaceWeb: a named namespace that can be referenced. In this proposal, one of the TWikiWebs will be the NameSpaceWeb.
• SubWeb (or perhaps Folder): a TWikipart of a web in the same namespace that is highly linked with other SubWebs in the NameSpaceWeb.

The goals of this extension are as follows:

• Allow a SubWeb to be associated with a NameSpaceWeb. Usually, this means that you can visualize that the SubWeb falls inside the parent NameSpaceWeb. The logical result is that users can specify WikiWord links without needing a TWikiWeb prefix when the topic lies in a SubWeb that is in the same NameSpaceWeb.
• Allow specified TWikiWebs to be logically global to all NameSpaceWebs. This is handy for documentation and administrative webs that should be logically available to all webs.
• Allow TWikiWebs to be isolated into separate namespaces, such that the web-name prefix is required. This is the same operation that we have today. The point is that you can mix the concept of some SubWebs that are within one NameSpaceWeb and still have other NameSpaceWebs with other SubWebs within that.
• Allow the specification of WikiWord topic names, to be extended across all SubWebs in the NameSpaceWeb without the Web prefix for:
  • Go-Box searching
  • Embedded WikiWords
• Allow the specification of WikiWord topic names WITH the Web prefix when desired and for backward compatibility.
• Search scoping automatically extended to all SubWebs logically within the NameSpaceWeb.

A goal that is not included is the option to have a SubWeb logically within more than one NameSpaceWeb, but not in others. In other words, a SubWeb is either wholly "within" a single NameSpaceWeb OR it is logically considered part of all NameSpaceWebs, but not "part of some but not others". It was felt that this feature contributed only a smidget of functionality for an inordinate amount of extra syntactical complexity.

Proposed syntax

Within the WebPreferences topic of the web in question, the following variable MAY exist:

• Set NAMESPACEWEB = Webname or "ALL"

Where:

• Webname is
  • the name of an existing web, considered its "parent". Default is the current web, OR
  • "ALL", which means that the current SubWeb is considered a part of every namespace.

Example

Assume that you want to have a three namespaces on a TWiki site called "Main", "Two", and "Sandbox".

NameSpaceWeb	SubWeb	NAMESPACEWEB Setting	Description
Main	Main	Main	Top folder would include master WebPreferences and other meta information about the web/namespace
Main	Help	ALL	user-oriented help information, tutorials, etc., but NO "blood and guts" documentation
Main	Docs	Main	"Blood and guts" documentation, Plugin interface, Plugins available, how to install, etc.
Main	User	ALL	current User web including user preference topics, list of Users, etc.
Main	Admin	Main	Current TWiki and admin tools, locked to user changes, accessible only by AdminGroup
Main	Groups	Main	Groups: Description of all groups, maybe list of users goes here.
Main	Templates	Main	All topic templates and include topics that are infrastructure only, locked to user changes
Main	Forms	Main	All forms topics that are infrastructure, locked to user changes.
Main	Plugins	ALL	All installed plugins, locked to user changes
Main	Codev	Main	User discussion
Main	Support	Main	User discussion
Main	Brainstorming	Main	User discussion
Main	Bugreports	Main	User discussion
Two	Two	Two	WebPreferences for this namespace
Two	Examplesubweb	Two	arbitrary use, but a secondary namespace is generally not needed for most TWiki installs
Sandbox	Sandbox	Sandbox	WebPreferences
Sandbox	User	Sandbox	arbitrary use, likely tests established by individual users.

Notes:

• The Help,Docs, and Plugins SubWebs are global and effectively part of all other webs.
• The WebPreferences and WebNotify topics will exist in every SubWeb and are an exception to the namespace merging, as these have the same name and are distinct.
• In this example, embedded WikiWord links will be found in any SubWeb that is part of the same NameSpaceWeb where the topic exists.

Narrative Example (Adapted from TopicNotFoundInThisWeb)

First, I will repeat the scenario as provided in TopicNotFoundInThisWeb:

Say someone were to work for a company called BigTimeConsulting (BTC) which had divisions for Audit, Tax, Legal and BusinessConsulting (BC). Every person in BC then belonged to: a group such as Technology, People or Process, and a specialism such as KnowledgeManagement or DataManagement an industry team Furthermore, each person then worked on one or more projects. Then, lets say that each of these areas (BTC, Audit, Tax, Legal, BC, Technology, People .... okay you get the picture .... all the way down to a project) gets its own web. As an organisation, BTC would want to appear coherent at the very least to the outside world. This would mean that: Terms would need to be used consistently and not duplicated between parts of the organisation. (This can be difficult once an organisation gets beyond workable grapevine size) We can say certain things about each web: Certain webs implicitly inherit concepts from others: BTC would need key (market-facing) concepts from each of the divisions You would use ImportingContexts to bring in the market-facing concepts from BTC into the divisions, without bringing in terms such as KnownProblems (see WhatDifferenceBetweenATopicAndAWeb) There would be an explicit hierarchy such that that which was intended by default would be the web that was closest to the current topic. All webs that form the hierarchy form the context for that topic. So when contributing topics to a web about KMProject for CTDI bank using middleware technologies for the knowledge management specialism in the Technology group in Business Consulting, my entire context would be (most-relevant to least relevant): KMProject CTDIbank middleware technologies knowledge management specialism technology group BC BTC Consequently, when topics in KMProject refer to a WikiWord TIBCO, I don't need to define the topic TIBCO or StandardTermsAndConditions in KMProject. Instead the KMProject web explicitly inherits the CTDIBank, middleware technologies and knowledge management specialism webs and through inheritance brings in the (exported) terms from technology group, BC and BTC. It might find StandardTermsAndConditions in BTC and TIBCO in middleware technologies

This is a great scenario and will be a good test of the proposal. I will use a table format to describe the settings a further generalized situation. I will assume that the various departments listed will want to have some stuff private and participate on projects in a matrix-style organization, which we may want to model as follows:

		Projects
Functional Department	SubDept	ProductA	ProductB	ProjectC
Admin
Admin	People	participant	participant	participant
Admin	Legal	participant	participant
Finance
Finance	Accounting	participant	participant	participant
Finance	Tax
Finance	Audit
Marketing
Marketing	Marcom	participant	participant
Marketing	Events	participant	participant
Marketing	Product Management	leader	leader
Sales		participant	participant
Sales	Inside			participant
Sales	Region1
Sales	Region2
Sales	Region3
Operations
Operations	Manufacturing	participant		participant
Operations	Parts
Operations	Facility			leader
Development
Development	Software	participant	participant
Development	Hardware	participant	participant
Development	Testing	participant	participant
Development	Requests

In this model company, each department will have some internal stuff that it does, and it will want to cooperate on projects that are common to some departments and not other. In this model, we have two products that are headed up by the Marketing Department, and one project that is headed up by the Operations department (changing the facility situation). Some departments will be involved in some of these projects, while not in others.

In the current proposal, for the sake of simplicity, if any of the SubWeb is "shared" with other webs, then all of it is. I will continue to use this in the example.

NameSpaceWeb	SubWeb	NAMESPACEWEB Setting	Description
Main	Main	Main	Top folder would include master WebPreferences and other meta information about the web/namespace
Main	Help	ALL	user-oriented help information, tutorials, etc., but NO "blood and guts" documentation
Main	User	ALL	current User web including user preference topics, list of Users, etc.
Main	Groups	ALL	Groups: Description of all groups, maybe list of users goes here.
Main	Templates	ALL	All topic templates and include topics that are infrastructure only, locked to user changes
Main	Forms	ALL	All forms topics that are infrastructure, locked to user changes.
Main	Plugins	ALL	All installed plugins, locked to user changes
Admin	Admin	Admin	Admin Department, general stuff
Admin	People	Admin	Admin Department, People subdepartment (Human Resources)
Admin	Legal	Admin	Admin Department, Legal subdepartment
Finance	Finance	Finance	Finance Department, general stuff
Finance	Accounting	Finance	Finance Department, Accounting subdepartment
Finance	Tax	Finance	Finance Department, Tax subdepartment
Finance	Audit	Finance	Finance Department, Audit subdepartment
Marketing	Marketing	Marketing	Marketing Department, general (internal) stuff
Marketing	Standard Terms	ALL	Standard Terminology used company wide
Marketing	Marcom	Marketing	Marketing Department, marketing communications
Marketing	Events	Marketing	Marketing Department, events, tradeshows, etc.
Marketing	Product Management	Marketing	Marketing Department, events, tradeshows, etc.
Sales	Sales	Sales	Sales Department, general (internal) stuff
Sales	Inside	Sales	Sales Department, inside Sales
Sales	Region1	Sales	Sales Department, region 1
Sales	Region2	Sales	Sales Department, region 2
Sales	Region3	Sales	Sales Department, region 3
Operations	Operations	Operations	Operations Department, general (internal) stuff
Operations	Manufacturing	Operations	Operations Department, Manufacturing
Operations	Parts	Operations	Operations Department, Parts
Operations	Facility	Operations	Operations Department, Facility
Development	Development	Development	Development Department, general (internal) stuff
Development	Software	Development	Development Department, software group
Development	Hardware	Development	Development Department, hardware group
Development	Testing	Development	Development Department, testing group
Development	Requests	Development	Development Department, change requests
Development	Bugreports	Development	Development Department, bug reports
Projects	Projects	Projects	Projects Web
Projects	ProjectA	ALL	Projects Web, Projects A SubWeb
Projects	ProjectB	ALL	Projects Web, Projects B SubWeb
Projects	ProjectC	ALL	Projects Web, Projects C SubWeb
Sandbox	Sandbox	Sandbox	arbitrary use, likely tests established by individual users.

Note that we may want to extend the syntax to "import" names from specific SubWebs instead of just using "ALL". I considered this extension and would like your comments as to whether the complexity is worth the benefit. The above scenario is quite involved, and yet most of the name scoping issues appear to be adequately handled. However, the following extensions are suggested if we think selective importing is appropriate:

• Set NAMESPACEIMPORT = Webname list

This would import a list of subweb names into the current namespace. This is akin to the typical namespace declaration used in XML.

Implementation Considerations

Upon initialization, it will be necessary to recover the information about the namespace scopes of the site.

• For a given SubWeb, it will have a NameSpaceWeb, which defaults to its own web name, or "ALL".
• Each NameSpaceWeb will have a list of SubWebs that are logically part of it.

When a search is performed,

• "ALL" will mean all webs,
• a SubWeb name means that specific SubWeb, and
• a NameSpaceWeb name means all SubWebs associated with that NameSpaceWeb.

When a Go Box request is actioned:

• default action is to search the current NameSpaceWeb.

When an embedded link is evaluated:

• it is linked to a topic within the current NameSpaceWeb.

When clicking famous ? link:

• create new page in current NameSpaceWeb
• preferrably allow user to select which SubWeb to place the link.
• no famous ? link will exist if the topic name exists anywhere in the namespace, so there is no conflict for new topics created.

For existing installations, there may be more than one topic with the same topic name in two separate SubWebs that were previously not in the same NameSpaceWeb. In such a situation, the first one found is located and the other one is disregarded (OK?).

Contributors:
-- RaymondLutz - 31 Dec 2003

---

Please insert your comments here

Hi Raymond, I agree for the need for this, but I have to admit to finding your example unclear. Is it possible you could draw some diagrams or add an example narrative? TopicNotFoundInThisWeb has an example narrative, perhaps you could adapt this one?

-- MartinCleaver - 31 Dec 2003

Raymond, please see the talk we had Martin and I on IRC: http://koala.ilog.fr/twikiirc/bin/irclogger_log/twiki?date=2003-12-31,Wed&sel=21-49#l17

and tell us if we misunderstood you and what you think of the "scope capture" issue...

-- ColasNahaboo - 31 Dec 2003

I think the following is the crux of the issue you are bringing up:

ColasNahaboo: I think that he says that if you declare the web "Support" in the "Codev" namesapce then in "Support", if you write KoalaSkin, it will automatically link to "Codev.KoalaSkin" if no "Support.KoalaSkin" exists ... the problem I see is that if at a later date, "Support.KoalaSkin" is created, you modify the meaning of existing pages... exactly the dreaded problem of "dynamic scope" in programming language that mnost modern languages search to avoid...

I don't think this is an issue because you can't create a topic with the same name in two SubWebs with the same name, as they are in the same namespace. in your example, it is not possible to create both Support.KoalaSkin and Codev.KoalaSkin -- only the one that is first created will be allowed. Making both will certainly be confusing to any user anyway. The only gotcha to this scenario is for sites that are changing from discrete namespaces to SubWebs in a single namespace, there is a temporary period where topics may have the same name in the separate namespaces and will conflict in the new namespace. I see this as a small issue that really won't haunt us for very long, and there may be some tool we would like to create that will inform us of identically-named topics in SubWebs so we can rename them and avoid the conflict and user confusion.

I modified the above specification slightly from the prior version and studied the TopicNotFoundInThisWeb description and adapted it above.

-- RaymondLutz - 31 Dec 2003

Simple problem with the example above:

• Initially this looks fine and dandy.
• However twiki webs often change (and emerge) by evolution, not revolution
• A conceptually simple change to the situation above is for the standard terms to migrate to admin for ownership, for a "developer terms" to evolve for the development group - which over time evolve to marketing and then admin.

In the above example it's tricky to see how this would be accommodated in a way the average user would use given we're talking about normal topics since it (appears) to work on the basis of a NameSpaceWeb acting as a set not as a bag. Thinking of merged namespaces as bags, not sets - since topics with the same name with different contents can make a lot of sense - is IMO the way to look at this.

That said: Performing web union (or more usefully, import) would be a very useful thing - often I've found many sites like wiki's and go "ooh, can I have one of those" - with the result being a proliferation of webs. Being able to reign these back into a central one would be useful - though I must admit I'm not convinced by this approach. (That said, code vs no code wins - if you write this, I'd probably use it )

Other uses of web union are:

• ApplicationWebs (include the contents of a web & its attributes)
• SeparateTWikiSystemAndSitePreferences
• IncludePreferences (union the attributes of a web, not its contents)

-- MS - 05 Jan 2004

Michael, I agree with you that the Wiki concept is a very dynamic and evolution is the way ideas develop. But, webs are not available to the contributors unless they have access to this sort of function, and for most sites, I would be willing to bet that the creation of webs is not available. Even on TWiki.org, there is a discussion about creating a new web for NextGeneration discussions. Indeed, allowing users to make webs, forms, templates and the like would be possible but restricting this to the administrators of the site, or at least a small set of persons is more likely.

I like your comment about Set vs. Bag for topic names in the same namespace. (But the list of webs in the same namespace are strictly a set). Bag can have duplicates that set cannot, and Bag obviously would be the case for situations like WebPreferences and other topics with standard names, and is a departure from the usual namespace methodology. I imagine this same concept can be extended to any other topic names, and the local name would be found if no webname qualifier were provided. Yet, when a new topic is created, a warning should be given if the name exists in some other web that shares the namespace with this one.

I have been cogitating over this proposal and have some problems with it myself, and think we can improve and simplify it slightly, while addressing these valid concerns. I wrote a concept on this but need to consider it further, so I did not put it in here. I will need to process it offline.

-- RaymondLutz - 05 Jan 2004

(playing devil advocate) Raymond, if you cannot have name conflicts, why not just use keywords (categories) on topics in a single big web specifying which "namespaces" they are in?

-- ColasNahaboo - 06 Jan 2004

Raymond, a web defines a scope for a view. It also defines a namespace. It also defines a default search. It also acts as a unit of systems administration. (OK in TWiki 3 units minimum...). WebForms do not have to be limited to web preferences - definition in user topics is also possible As I have done (temporarily) for this topic using a compatible form - and used this to change the classification slightly.

Why can't I have the following ?

• http://twiki.org/cgi-bin/view/Codev/FeatureEnhancementRequest/
• http://twiki.org/cgi-bin/view/Codev/FeatureEnhancementRequest/CairoRelease/100%/
• http://twiki.org/cgi-bin/view/Codev/Main.WillNorris/ (Sorry, picked the only name listed in InterestedParties )
• http://twiki.org/cgi-bin/view/Codev/InterestedParties/
• http://twiki.org/cgi-bin/view/Codev/TWikiRelease01Feb2003/
• http://twiki.org/cgi-bin/view/Codev/TWikiRelease01Feb2003/Summary/ (limits view of topic text)
• http://twiki.org/cgi-bin/view/Codev/TWikiRelease01Feb2003/title/ (limits view of topic text)
• http://twiki.org/cgi-bin/view/Codev/TWikiRelease01Feb2003/?bookview&summaryformat=SomeTopic

These require adhoc creation of webs, not adhoc changing of storage of webs. etc. These all form collections of interlinked topics. (ie webs) They boil down to providing a search mechanism through namespaces/webs -- but that's all TWiki webs are at present - especially when combined with skinning. This approach doesn't preclude your approach since essentially it's a searching system, but does show the power that putting web creation in users hands could provide as long as it's handled appropriately.

Bags in namespaces are not a new idea (or even that much of a departure) BTW, multimethods, multiple inheritance, weak entities, repeating fields, etc are not exactly cutting edge TWiki is data rich, such environments often end up with duplicates. How you deal with duplication, not how you enforce it's destruction is important. Consider people deciding to merge two inappropriate namespaces - it might not be something you're interested in at present, but it's something that someone will need to do at some point, and allowing this to happen relatively gracefully down the line is a good idea.

As I said before though, whilst I've got reservations on some details I like the basic tenet of the idea, and would be interested in an implementation if one were written

• Michael, please use the Sandbox web for form testing, thanks -- PeterThoeny
  • Peter, I was demonstrating that WebForms are not limited to administrators even if WebPreferences is only edittable by admin - as you should note from the context of my post. It was not testing - if I wanted to do that I would do so on my own servers - as I'm sure you're aware. I could do so and put a pointer there instead, but I suspect you'd prefer me not to do so based on your previous comments. I'll leave this meta discussion in since you've broken the context. -- MS - 07 Jan 2004

-- MS - 06 Jan 2004

Comments from any Devil's advocates are welcomed. I call this throwing rocks and any proposal can benefit from it. Before I respond to your comments, I want to make a point:

The concept that we wind up with regard to the set of topic names in a namespace is not a general Bag since:

• Topic names in a single web can't use the same name demanded by the uniqueness of file names in the file store. You can't continually add items of the same name a given web, but you could to a general Bag.
• You can only get duplicates if you have a topic of the same name in another web, such as with WebPreferences.

Therefore, conceptualizing this as a Bag is not the proper model. I assert that there is little reason to allow for duplicate topic names in a given site, unless you are doing something like providing virtually distinct sub-sites. The only exception is an artifact of how the settings are maintained for a web, i.e. as a WebPreferences topic in that web. These administrative topics that could be maintained outside the web as "out of band" information would indeed be exceptions to the requirement of unique names.

If two webs that were in separate namespaces are merged into a single namespace, you would ideally like to deal with the duplicate names. I see no problem in allowing duplicate names in different webs in the same "namespace" and these would require the use of a qualifier. It's like a join of two SQL records. You can have fields with the same names. If the fields have the same names, you must include the source table name as a qualifier. If not, just the field names will do. But, the topics are more correctly considered as records in a table. If we intend to allow TWiki to migrate to an MySQL implementation, we should consider the topic names as the keys to the records in the table, and the tables are the separate webs. The WebPreferences stuff would clearly belong outside of the table itself. If it and other admin stuff that exists for all webs, then the concept of a Set is a bit more clear.

Responding to ColasNahaboo, indeed, you could organize your site using a single namespace web and use other OrganizingPrinciples to sort through them. This is entirely supported, but there are other reasons to organize into separate webs, such as:

• Limit the scope of searches so that you are not looking through topic files that are not relevant.
• Allow for notifications to topics changed in a given web.
• Allow for differing access rights to topics in different webs through global settings.
• Allow for different sets of topic templates and forms in different webs.

As you can see, there are many reasons that you might want to use this other infrastructure that is associated with "Webs" regardless of the disposition with regard to the extent of their namespaces, and tagging can't to these. This is in fact the reason for this whole discussion. Indeed, it arises for me from a need to use several different default topic templates, and now only one can be associated with any single "web".

Michael, your change of the form causes me problems... I see not TopicClassification. Changing the form associated with a given topic is something that I see as a little-used and dangerous feature to provide to the public at large. There are many assumptions in play when constructing one of these TWiki forms and allowing anyone to change it will result in the same damage that you have done here. Please set it back.

On your example, you are equating the URL structure with the structure of information, which can be far different. Your examples DO NOT require the adhoc creation of webs. I think I am going to need some specific questions that you feel are important rather than listing some URLs that are supposed to explain your point.

-- RaymondLutz - 06 Jan 2004

Raymond I've discussed it all on this web before - I've put pointers, details and examples on page you reference. I'm not about to dig it all out again . I was pointing out real problems with your scenario which you appear to be missing completely(or ignoring). You're essentially merging namespaces, and you either deal with multisets or bags in that situation - whether you like it or not. Unless you can cope with people's existing webs enforcing the set nature of a namespace web is the surest way to ensure that no-one with a running install and with 2 or more webs actually uses the feature.

I've said that the feature you're proposing is interesting, useful and if implemented I would use, and then probably extend in other ways. If you want more of an explanation look at the TWikiIRC logs and NameSpaceWeb.

Key point: Wiki is designed to limit the problems caused by single webmaster syndrome (or as I prefer it, stop one person or group of people being a bottleneck or cause problems if hit by a bus) by granting all users the ability to correct all content. Limiting changes to WebForms, WebPreferences, structure, the ability for people to use the same name in different groups for different things (eg DRM digital radio mondiale vs digital restrictions management) and so on is the surest way to ensure that people set up their own systems. Multiple TWiki webs on a single server limit that problem since different policies can exist in different webs. TWiki might be usable (just) as an web application platform, but it's biggest single strength it shares with all other wikis - it's a wiki.

Namespaces - essentially a method of merging physical webs - whilst useful should bear in mind that the default for the majority of wikis is loose control, not tight. (Most don't require login, let alone registration, for edit, etc - TWiki.org is unusual in the severe level of control imposed here - something I find ironic)

In that scenario an evolution could easily occur whereby someone has:

• http://www.server.com/Devweb/UsefulLinks
• http://www.server.com/Devweb/Induction
• http://www.server.com/Devweb/Calendar
• http://www.server.com/Marketing/UsefulLinks
• http://www.server.com/Marketing/Induction
• http://www.server.com/Marketing/Calendar
• http://www.server.com/Shared/CommonTerms
• http://www.server.com/Shared/VacationBooking
• http://www.server.com/Shared/VacationBooking
• http://www.server.com/Shared/UsefulLinks
• http://www.server.com/Shared/Induction
• http://www.server.com/Shared/Calendar

Where shared is listed as being part of namespace web all. Unless you treat things as a bag, multiset, sister site, bookview, whatever, then you are limiting the way groups of people inside a single namespace can collaborate - since the Marketing & Development pages matching the shared web would be clobbered. Consider also that this is the sort of thing you do see in a running TWiki server inside companies - especially if people start running multiple servers and distribute content & merge across servers)

The ability for adhoc merging of webs, and even better content - ala book view - between Marketing & Dev might be important to people arranging meetings. You've argued elsewhere for the eradication of webs and to simply use searches - which is what an ad hoc web is. Now you seem to be arguing the other way - no ad hoc webs - only physical. The ideal IMO is to allow both.

Consider a dictionary/hash that can only contain key/value and values can only be atomic/scalar - rather than arrays/lists or hashes/dictionaries. (Or structs that can only contain one scalar per field) You can do things with that, but you need at minimum keys, joins and significant levels of control of what the user puts in things - moving away from the basic wiki nature.

TWiki is a database (using the mini-world definition) that already allows duplicate keys and deals with unstructured data as well as structured. Webs are subsets of that data and due to filesystem constraints don't contain duplicate keys. Merged webs (static as you propose, or dynamic as I suggest) do not have to have the same constraint and if they did you introduce hideous problems for users. If you still can't see the problems (as well as the clear benefits that DO make the feature worth starting) I'm clearly not getting the idea through - and I'll leave it to someone better capable to do so if they think it's worthwhile.

-- MS - 07 Jan 2004

Well put, Michael, I agree. The NameSpace concept needs more thought; it's not obvious how to get there from where we are today, and ad-hoc solutions will only make the problem worse. Personally I think we need to bite the bullet, flatten out the data directory and index it properly with a database.

-- CrawfordCurrie - 07 Jan 2004

As usual, I have a time at about 3:30 am when my mind rolls over solutions to problems I am working on, whether I like it or not. Several years ago, I decided to embrace this interruption of my sleep... I now consider it my ally rather than my enemy. Well, last night, I think I resolved the proposal and will try to described the changes here. But first, I want to respond to some of the other points.

• I am not trying to support one way of doing things that is different from the way things can be done now. I am not trying to impose any more control than seems appropriate for the TWiki being developed, only provide more options, in areas where I want to make progress in real situations, and don't have the tools to do these things now in the current structure.
• Many TWiki sites embrace the loosey-goosey approach, but these approaches are NOT always appropriate. If you want to allow for other sorts of applications, you have to ALLOW for other OrganizingPrinciples.
• Regarding treating TWiki as a database, indeed it is but one which is highly constrained and does not provide the wealth of capabilities as even the most primitive of databases. I have already had trouble with using the formatted text in the body approach as people don't respect the format enough for this to be a reliable method.

With that said, I have the following suggestions for changing the current proposal. I will make these suggestions here before I refactor the proposal section.

First, the key issue is that we need to separate two distinct issues:

• Creating topics within a namespace.
• Searching for topics within several namespaces.

One definition to help us, that is the WebMetaTopics. These topics contain data that describe the web itself, such as WebPreferences and WebNotify topics, and must be specially handled, and they are today.

I again like the concept of the SubWeb, in that it:

• lies strictly within the namespace of a named "parent" NameSpaceWeb.
• inherits the settings of that web unless overridden by settings in its own WebMetaTopics.
• complies with the concept that the topic names amongst all the topic names in the NameSpaceWeb are a Set (i.e. no duplicated names), except for the WebMetaTopics.

The SubWeb concept handles the issues of creating a new topic. You would be restricted from creating duplicate topic names in a single NameSpaceWeb, and its SubWebs.

Therefore, the change proposed to the proposal and its description above with regard to this first issue is that the concept of "ALL" is not allowed. Only a single NameSpaceWeb is allowed as the operand to NAMESPACEWEB.

Next, the issue of searching for topics. We want the search to proceed over more than one topic, presumably generally including the documentation and TWiki, webs, but not necessarily. To implement this, I think something along the lines of the SearchElsewherePlugin is appropriate. I think your idea, Michael, of importing the scope of the allowed search is probably the approach with the most facility, and "ALL" would then mean that you are importing all other webs for a search, and listing the webs that are included would generally suffice. For this purpose, an additional setting like

• Set IMPORT_TOPICS_FROM = list of NameSpaceWebs or SubWebs

If this is clear enough from this short description, I will proceed to modify the proposal above accordingly, as I don't like it as it was earlier written.

-- RaymondLutz - 09 Jan 2004