• 22 Dec 2000: TWiki (RCS replaced by ClearCase) by MikeRovner for internal use (behind a firewall) as project documentation tool

TWiki will be modularized at some point so that the backend storage can be replaced easily, for example by ClearCase. ClearCase is the standard CM tool for large corporations, just because of that it would fit the TWiki mission as an alternative to RCS.

Question: Besides being the corporate standard, what are the benefits of using ClearCase instead of RCS in TWiki?

-- PeterThoeny - 29 Dec 2000

First, let me pinpoint some benefits of the usage corporate standart:
- you have all your code and documentation in it, that provides easy access to all your stuff,
- it's usually already covered with backups procedures.

Than (not implemented yet), ClearCase provides good automatic merging capabilities, allows to edit a doc simultaneously and then automerge both edits with automaticly inserted appropriate authors marks.

However there are some differencies in both CM and usually ClearCase is more restrictive in version deleteing though it's preferable to change (as I did) version saving policy in TWiki to replace last version in place and check it in only on "Release edit lock" mark set.

Second big issue is access control. ClearCase allows do a lot in that direction. However Twiki is (now) mostly used as an internal group collabaration tool and doesn't need such advanced features. However they would be great for putting Twiki on the next level.

-- MikeRovner - 02 Jan 2001

I was wondering whether anybody has looked into using ClearCase instead of RCS for hosting TWiki. I am interested in this for two reasons:

• ClearCase is the "approved" configuration management system in our company
• ClearCase multisite is a simple means of distributing a TWiki over a number of sites.

The latter feature interests me the most. I am trying to have a number of sites world-wide work on TWiki. I am afraid that the bandwidth limitations that we have to some of the sites makes it difficult for them to work of a central TWiki server. I am considering using the replication mechanism of ClearCase multisite to replicate the TWiki repository to each site so that they are working local but are synchronized if the same Web is changed.

I anticipate that it will be rare that the same web will be acessed in write mode by more than one site.

-- ThomasWeigert - 29 Jun 2002

That sounds like it should work, though I must share our experience of using ClearCase as a backend for TWiki.

ClearCase is also the "approved" configuration management system at our company, and I had modified our TWiki (and our web server, to run in a view) to use it.

After using it for a few weeks, the users noticed how slow it was at retreiving pages and doing searches, which involves opening large numbers of files, even in a local (non-multisite) vob. In a side-by-side comparison with RCS, we noticed at least a 4-fold difference in speed in favor of RCS.

As you implied, speed is critical for a good user experience. In general, ClearCase is not known for it's wiki-ness when large numbers of files are involved. Seldom accessed topics are also very slow when initially accessed (in addition to topics accessed for the first time in the morning, for some reason), and require at least one "wake-up" access for ClearCase to pop the file into the web server view's cache.

What we did as a compromise was store various large documents and design information in ClearCase vobs (for our everyday work), and make them accessable via a ClearCase plugin (yet to be posted).

ThomasWeigert : I anticipate that it will be rare that the same web will be acessed in write mode by more than one site.

Though you don't anticipate multi-site edits in the same web, it may happen... in which case you may need to answer these questions: Am I going to run each TWiki in a distinct config spec? Am I going to allow edits for each site only in the web belonging to that site (This can be done by defining groups for all the members of each site, and using them for access control)? Will each site have it's own branch, and merge back to the main branch for each checkin? Will we merge from the main branch to the site branch for each checkout? How do we plan on dealing with merging the results of several simultaneous edits back to the main branch from each site?

Sounds pretty gnarley...

-- PeterNixon - 30 Jun 2002

I believe that you are right in your general assessment. I was primarily hoping to use multisite for replication. I was indeed curios about whether somebody had experience with ClearCase as a backend. I am not surprised by your observations... there is a lot more going on when updating a VOB then what RCS is doing. Nevertheless, if you do have a plugin for ClearCase, I'd appreciate your posting it (our situation is similar to the one you describe... there are documents that we need to pull out of a VOB).

-- ThomasWeigert - 30 Jun 2002

Interesting point about performance - searches on standard TWiki never use RCS, they just open the *.txt files, so if there was a way of keeping the *.txt files with ClearCase you would see a dramatic improvement in search performance.

To support a global user base with limited WAN bandwidth, you might want to look at proxy cache usage with TWiki (most companies with such setups already have such caches) - see BrowserAndProxyCacheControl. TWiki already has some code to set HTTP headers (in the fix to BackFromPreviewLosesText) and you could just write a plugin for the TWikiAlphaRelease to set suitable cache control headers - e.g. you could set all pages to an expiry time of (say) 4 hours, greatly reducing the number of accesses over the WAN. This assumes that the number of views is much greater than the number of updates, which is usually the case.

See also TWikiWithClearCase, and ReadWriteOfflineWiki and MirrorSite for some discussion of distributed TWiki approaches - a TWikiContributor has the latter working at their workplace, I believe.

-- RichardDonkin - 30 Jun 2002

The interesting difference between RCS and ClearCase is that ClearCase is a virtual NFS file system, which can be mounted almost the same way as any other remote NFS file system. In order to access any file (at all), you need to be in the virtual file system's directory structure. ClearCase intercepts any file system calls (such as open, close, flush, etc) which affect files in that directory structure to actively piece together the requested file from a differences database and present it to the user. File accesses are cached by ClearCase, but not for long, especially if the database is very large and there is a lot of file activity.

Files and directories may be added to the revision control system individually, and the visible version of each file or directory is selected by a set of labels in a configuration specification. You can change the version of the file you're looking at just by changing a label in your config spec.

The overhead associated with ClearCase's interception of all file system calls to see what it should show the accessing program (even for an ls command) can be prohibitive, especially if there's a cache miss. We tend to run programs which necessitate large amounts of file access (even for printing out a log file) in a real NFS partition, and check the results into the revision control system after the job is complete.

The ClearCase plugin is included with MegaTWiki, but I will make it a separate package, along with the other bits that can be separated out.

-- PeterNixon - 30 Jun 2002

twiki-clearcase-bin.tar.gz (Contens of twiki/bin modified for ClearCase)

To add to the points above, ClearCase would allow for easier distribution of TWiki using its multi-site capabilities. There are issues with the use of ClearCase regarding performance in searching that have been pointed out in TWikiOnClearCase (these topics should be merged together).

-- ThomasWeigert - 30 Jun 2002

Administrative note: Merged TWikiOnClearCase and TWikiWithClearCase topics.

-- PeterThoeny - 07 Jul 2002

I am not sure I see the benefit of clearcase here.. In terms of feature set, if 'merge' or 'mirroring' or 'web access to revisions' or 'account management' is desired, why not use cvs with cvsweb instead, which is free beer source.

I suppose the remaining argument is that clearcase is the 'standardized enterprise app'. And that clearcase has an expensive gui frontend. Ah, expensive, let me say that again: expensive.

Though from a development standpoint if the code is developed to allow both clearcase and cvs (and what next, mks?) then all the better..

-- TWikiGuest - 07 Jul 2002 (jcline at ieee.org)

I wonder if there is possibility to get update of twiki-clearcase-bin.tar.gz to be working with the latest version of TWiki?

-- JacekZapotoczny - 28 Apr 2005

I'm also wondering if there could be an update of twiki-clearcase-bin.tar.gz to work with the latest version of TWiki. Also the file attached to this topic seems to corrupted, through Cygwin it states that the archive contains obsolescent base-64 headers and doesn't look like a tar archive!

-- SouravMishra - 22 Jul 2005