Hierarchical Access Control

From the TWiki-04 there is, embedded in the system, the possibility to create a hierachical tree of webs with settings and permissions that inherits from parents tu sub webs. This is a great opportunity for large intranets and customer/supplier oriented istances, but it seems to be some big problem with the access control management.

Expected Behaviour vs Reality

For example, in a typical customer/suppliers oriented istance we would like to obtain a behaviour like that:

Customers

Customer1

Customer2

Customer3

Customer4

The expected access control would do:

• the supplier can have access to any web to create an index on Customers (and maybe to comment anything as he want) and to populate the Customers/CustermerX with documentation, price lists or whatever.
• Customer1 can have access only to Customers/Custermer1 web, the supplier gived him an URL thar point directly to that. The same to other customers.

We should obtain this result simply using ALLOWWEBVIEW = Main.SupplierGroup in Customers/WebPreferences, ALLOWWEBVIEW = Main.SupplierGroup, Customer1Group in Customers/Customer1/WebPreferences, and so on.

In the reality this is not achievable, we can obtain the following cases:

• ALLOWWEBVIEW = Main.SupplierGroup in Customers/WebPreferences, ALLOWWEBVIEW = Main.SupplierGroup, Customer1Group in Customers/Customer1/WebPreferences, and so on: the customers can't access to related webs.
• ALLOWWEBVIEW = Main.SupplierGroup, Customer1Group in Customers/WebPreferences, ALLOWWEBVIEW = Main.SupplierGroup, Customer1Group in Customers/Customer1/WebPreferences, and so on: customer1 can access to his web but can also (obviously) access to the Customers web AND (less oviously) to any other CustomerX web.

-- Contributors: IvanSassi

Discussion

This is a great limitation of flexibility of the TWiki enviroment. The correct management of hierarchical webs is a key area to work on for using it in large infrastructures.

-- IvanSassi - 08 Aug 2006

Did you remove the ALLOWWEBVIEW from the FINALPREFERENCES in Customers/WebPreferences? If you leave it in, ALLOWWEBVIEW cannot be redefined at a lower level.

-- PeterThoeny - 08 Aug 2006

Ivan, you will have seen the rules in the documentation which TWiki follows for resolving access rights. To move this forward, how about you rewrite the rules here taking into account your requirements. That way we will have a sensible basis for discussion, and ultimately for writing the required tests.

-- CrawfordCurrie - 09 Aug 2006

Peter, no, I haven't tried cause the FINALPREFERENCES should works at topic level and I haven't thought that it could apply ever at web level (my fault). As you expected removing the parameter permit to give correctly the access to subwebs. However maybe this give too much control on the web to users, so we should consider it as a workaround?

-- IvanSassi - 09 Aug 2006

Crawford, the TWiki evaluation model is correct for me, and with Peter suggestion it works as expected. The only improvement achivable, at this point, is to study a way to separate the overriding rights rules in topic level and web level.

-- IvanSassi - 09 Aug 2006

That is a good point. Taking ALLOWWEBVIEW off of FINALPREFERENCES allows users in the current web to override the setting, probably not what you want! Not sure what a good spec is. Ideas?

-- PeterThoeny - 10 Aug 2006

How about keeping the current behavior of FINALPREFERENCES and adding a new pref called something like WEBFINALPREFERENCES which only applies to the current web and isn't inherited? In TWiki.TWikiPreferences we'd need a preference (call it NOINHERIT) which lists the names of variables which don't get inherited...

-- PeterNixon - 11 Aug 2006

Rather than hacking more complexity into the preferences mechanisms, I would far prefer we focused on taking ACLs out-of-band.

Out of band ACLs would allow a protection scheme similar to UNIX file/directory permissions, which is far more natural for a hierarchical web structure. A quick hack fix will just confuse the issue further, I fear. Note that out of band ACLs are already required for other reasons (protecting old revisions, principally)

-- CrawfordCurrie - 16 Aug 2006

The out-of-band has been discussed elsewhere. If done, it complicates matters further because the access settings are no longer automatically version controlled (a complete audit trail is a must in a corporate environment.)

The hierarchical web feature is not KISS (as stated in the TWikiMission), LogicallyNestedWebs would have reduced the complexity. Since we decided to take hierarchical web into TWiki we need to bite the bullet and make it work properly.

One simple solution to the current problem is to change the spec of FINALPREFERENCES so that it is a N/A for subwebs, e.g. take effect for topics, but not subwebs. This has one drawback that you cannot finalize settings on a higher level web for child webs. I think this is a small price to pay in order to keep the spec simple.

-- PeterThoeny - 16 Aug 2006

it complicates matters further because the access settings are no longer automatically version controlled - sorry, but that is the whole point. By versioning the access controls, you make it impossible to change access controls on old versions. Thus, if you accidentally revealed the corporate sales plan for next year, your only way to retrospectively hide it is to completely delete the revision; you cannot make it readable only by the board. Deleting the version obvious breaks the audit trail. AFAIK taking access controls is the only way to solve this. Given that, I see OOB access controls as a requirement, not a nicety.

Changing the spec so that FINALPREFERENCES behaves differently depending on where you use it, does not sound like a particularly simple, or user-friendly, solution to anything, to me.

-- CrawfordCurrie - 17 Aug 2006

Access control of versions

On access control spec I agree 100% with Crawford. IMHO the only logical and reasonable way they can work is like this.

• The access control in the current version applies to all old revisions.
• The access control in old revisions can be read by the naked eye in an audit trail but they do not open or protect the old revisions.

You may experience users that say they want something different but then they have not thought enough about what the consequences are.

Let us look at the some typical use cases.

Old revision contains secrets, current revision does not

Here you could propose that old revisions could be protected if they contained an ALLOWTOPICVIEW. But in reality, can the user be sure that the exact old versions with the secret are protected? No! Maybe revisions 1 and 2 are safe. Revision 3 has a secret but the user did not set ALLOWTOPICVIEW. In revision 4 a user added ALLOWTOPICVIEW. In revision 8 the secret is removed and the ALLOWTOPICVIEW is removed or altered. The users cannot change the access rights to the older versions. Such an implementation is not only confusing. It is also dangerous. The users will never have a chance to do a fair evaluation if exactly the versions with secrets are the ones that are protected. There could be 150 revisions of the topic. There will often be revisions that have the secret and are not protected. The users need to know with a simple rule that if a topic once contained a secret, you need to maintain ALLOWTOPICVIEW forever. If you need to allow access to only new information you must do it in a new topic, or delete/rename the old topic and create a new topic the old name. Anything else is unmanagable.

The current topic contains a secret - the older do not

Should the previous topic versions be accessible? No. I am sure that the logical and intuitive expectation of a user is that if I disallow topic view on a topic then the entire topic including old revisions is locked. If you want the old revisions to be accessible and new info to be secret you create a new secret topic. Anything else is not intuitive.

Latest topic version is secret - previous versions were also secret but we had forgotten ALLOWTOPICVIEW

This is a common situation. Someone put something secret. Someone else discovers this and add or alters the ALLOWTOPICVIEW. If ALLOWTOPICVIEW in older revisions are active, then you can never close access to the secrets. Your only chance is to delete the topic and ask an admin to wipe it from the Trash web.

No matter how I put up examples - it is either dangerous or lack intuition. If you lock the door the house is locked. If you unlock the door the house is open. That is the only logical way that things can work - like they work today.

In-band vs out-of-band

Crawford is seeking a performance gain by having access controls out of band. Peter is seeking audit trail by having in-band.

You are both right!

The good old ALLOWTOPICXXX that can be anywhere in the topic is actually a feature I like. I often want the access control very visible. Especially ALLOWTOPICCHANGE. Users know who to ask for access. Anyone with access can grant access. That is the wiki way. Let us not loose that.

I also like the new META way of adding access control. The in-topic access control has a good audit trail. I can see who and when access is changed.

To get the performance increase the access control should not be read from topics. Having to regex through many topic during searches must be a major performance killer.

So let us have both! In a smart way that both give performance gain and maintains audit trail and flexibility.

This is how it could be made to work

When you save a topic the topic text and meta data can be parsed for access control and the settings stored out-of-band. The text in the topic itself stays leaving the audit trail. But the access rights data is always read from the OOB file.

It is scanning for access control in many topics that takes the time and kills performance. Besides scanning the current topic when you save the ALLOW and DENY settings and META equivalents are simply ignored. The parseing we have already. The OOB handling we would need to implemented anyway. I think this is a realistic way to implement it.

Left is the issue of what to do with compatibility. That will probably need some more thought.

If all topics have an entry in the OOB access control file, then the perl code can see if a topic is new or old. If old it is parsed the old way for access rights. When topics are saved they becomes "new" by having their entries added to the OOB access file. This way an upgrade script is not required. But an upgrade script could be still be provided that updates the OOB access control file. And with our new configure such function could be a button you press in configure that starts the script.

Access control for site level, web level and subweb level would also be added to the access control file when saving the TWiki-/Web- Preference topics making protection of subwebs easier to manage in code I am sure.

The nice things about this compatibility mode are:

• Easy upgrade
• You can still simply copy and move and delete topic files at command prompt level. Rebuilding the access control index would only be required when moving files, and only recommended for performance when adding/deleting files. The same script that removes sessions can do this periodicly.
• You can still hack anything directly in text files.
• In line with the TWikiMission
• Performance boost to TWikis with many topics and many users (where we need the boost)

I am sure there are things I did not think about but I hope this brain-storm will inspire.

-- KennethLavrsen

Having the access control automatically parse topic not already in the OOB is also good for being able to drop the txt files into the data area without having to worry about adding access controls. There are probably many senarios where that is happening on current TWiki installations.

-- SamHasler - 17 Aug 2006

Note that the WebDAVPlugin parses topics and stores access controls out of band (for use by the Apache module) in a TDB database. This code could be lifted almost verbatim, should anyone wish to do it. Note however that TDB would be required.

-- CrawfordCurrie - 20 Aug 2006

Whatever the implementation, I'd prefer to see it done in some easily repairable, easily parsable plain text format.

-- PeterNixon - 21 Aug 2006

A disadvantage of OOB is that it can get out of sync with the topic store, that is it may still contain data for topics that don't exist anymore or have been renamed.

-- MichaelDaum - 21 Aug 2006

Sorry guys, but what is OOB ?

-- KeithHelfrich - 21 Aug 2006

PeterNixon: good point, though of course the tradeoff is in performance. What I had in mind was a side file that lists permissions for each rev.

KeithHelfrich: OOB means "out of band" i.e. not stored within the data of the topic, but stored elsewhere so that changes to the permissions are not registered as changes to the topic itself.

-- CrawfordCurrie - 21 Aug 2006

Crawford, are you thinking of having one side file per web? Can it exist as an editable topic, or are we thinking of a file living as an attachment somewhere in .../pub/web? If the format is compact and easy enough to parse, it shouldn't take very long to load with a good regex. Of course, there's still the performance hit of resolving TWikiGroups down to users.

-- PeterNixon - 21 Aug 2006

I was thinking of a file per topic, stored alongside the topic, and containing the same information as is currently embedded in the topic text. For example, MyTopic.meta. The format of the file would be very simple, and loadable using an eval (i.e. generated by Data::Dumper). For example,

$perms = {
    '1.1' => { 'ALLOWCHANGE' => 'LanceArmstrong', 'DENYVIEW' => 'JanUlrich, IvanBasso' },
    '1.2' => { 'ALLOWCHANGE' => 'OscarPereira', 'ALLOWVIEW' => 'FloydLandis', 'DENYCHANGE' => 'FloydLandis' }
}

If a topic has a .meta file, then it would not be parsed for permissions (the .meta file would be loaded instead). Otherwise a .meta file would be generated on the first subsequent save of the topic.

-- CrawfordCurrie - 22 Aug 2006

MicahelDaum: Oh, you mean like the .txt file can get out of sync with the RCS file?

Now how could that happen? It could be that someone bypassed the code that controlled how they were kept in sync and edited it directly ...

Now suppose we are talking about a regualr file system. The access control and time stamp and file map are OOB. They exist in the i-node rather than in the file. It is the job of the code to keep them in sync, but they can get out of sync if you bypass the operating system and patch the disk directly, altering the i-node or modifying the file without updating the timestamps or the map.

Convesely, if information is 'in-band' it can, just like the access controls in TWiki up to Version 4.0, be modified along with the content becuase it is part of the content - although it provides a structural function!

The moral here is that OOB is not evil or more risky. What matters is that the code for updating the in-band and out-band works proprly and is not subverted. This is the case with the RCS revision store in TWiki -- unless you bypass the code and edit the files directly.

My *-NIX file systems work quite well storing information out of band because I don't go bypassing the OS and patching the disk directly. I hope you don't succumb to the temptation to bypass the code and get things out of sync.

-- AntonAylward - 23 Aug 2006

Just had a random thought about performance; we're adding yet another file for TWiki to load (albeit a fast one if we just use Data::Dumper). Technically we already have the information in memory that we need for the majority of cases (latest version of the file)before we go to load this file via the basic preference loading mechanism (which also needs a performance boost). The topics already get parsed, WebPreferences already has been parsed, and so on up to TWikiPreferences. I would propose that this extra file be loaded only when access to a non-latest version is required.

On the other hand, we could take this concept a few steps further and apply it to preference loading in general.... Then we'd never have to parse anything more than the target topic text (as well as any included topics) on any given topic request.

-- PeterNixon - 23 Aug 2006

Not so random, these are good questions.

We have to think about several things;

1. what meta-data is taken out of band,
2. where we store the OOB data,
3. when we load the OOB data.

Let's consider the current situation: TWiki already stores meta-data out-of-band, specifically it stores revision info, creation date etc in the ,v file associated with each topic. (While TWiki attempts to duplicate this data in-band by also storing it in-line in %META fields, it frequently gets it wrong, and the approach is horrendously overcomplex and prone to the sort of errors alluded to by Anton, above). Given that we already have a place to store OOB meta-data, then it might be cool to leverage that place for other OOB meta-data. I was initially thinking of storing the ACLs encoded in the revision comment field inside the ,v file. However that has two problems:

1. It again locks ACL revs to text revs, so no gain there
2. The external invocation of the RCS programs required to access the meta-data is relatively slow, compared to reading the side file

Hence the .meta file. Now, when does the .meta file have to be loaded? For ACLS, only when a rev other than the latest rev is accessed. But what else might the .meta file contain? Potentially other revision specific meta-data. One example of this is an "elision flag". Some time ago SvenDowideit pointed out that deleting revisions out of the revision history was a major violation of audit principles, and suggested that instead of editing the revision history, TWiki ought to be able to hide specific revisions in the history (create gaps in history, without ever deleting any changes). The .meta file makes this possible. Another potential uses is for tags (in the TagMePlugin sense). I'm sure other people can see the new vistas of opportunity this presents. Because .meta uses Data::Dumper, extensions to the format are no-brainers.

Note that I do not favour taking form meta-data OOB in the current store. This is because forms should be revisioned in lock-step with the rest of the topic. However I do strongly favour removing META:TOPICINFO and the fields of META:FILEATTACHMENT that duplicate metadata already stored in RCS.

-- CrawfordCurrie - 23 Aug 2006

One advantage of OOB access control that hasn't been mentioned: The convnetion seems to be to put the ALLOW/DENY at the end of a topic and since topics grow that means the whole of the the topic has to be read in and the metadata pulled out before deciding "oops! we shouldn't be reading this in the first place". Or whatever.

And if you're not keeping all of the TWiki store in a database anyway, I'd use tied hashes. Extentions to that are no-brainers as well. I experimented with this back in the Bejing days and was impressed with its simplicity.

-- AntonAylward - 23 Aug 2006

So far folks have only mentioned the meta file which corresponds to the topic. To get around the "oops" problem Anton describes above, we'd have to have also read in the web's meta file (presumably WebPreferences.meta), as well as TWikiPreferences.meta, to have a complete view of access permissions. Crawford, is this what you are envisioning?

-- PeterNixon - 25 Aug 2006

Yes. I have in mind to store in the .meta files exactly what is currently stored in-band in the topic text, simply because it's easier for people to get their heads round that way.

-- CrawfordCurrie - 26 Aug 2006