We currently have an "UpgradeTWiki" script that claims to upgrade TWiki from older versions. It was a great idea at the time, but suffers from a couple of major problems:

1. No-one is maintaining it, so it is badly out of date
2. It's hard to test in all possible configs

I propose we get rid of it. The strategy I have adopted in 4.2 for handling upgrades is:

1. All compatibility code goes in TWiki::Compatibility
2. TWiki::Compatibility is loaded only on demand - where TWiki detects something that needs upgrading
3. Those upgrade steps that can be done in configure are done in configure, effectively upgrading "on the fly"

-- Contributors: CrawfordCurrie - 21 Jun 2007

Discussion

Yes, better to remove the UpgradeTWiki script, it has caused more trouble than good due to not getting maintained. (I renamed this topic to a more neutral name from "DitchUpgradeTWiki", let's be sensitive to code of other contribuors.)

Using configure to upgrade the TWiki engine is a sensible approach. Crawford, I remember that you did not like the idea of using configure for anything else than pure configuration, what is the reason to change your view?

This has also security implications. Does that mean that all libs need to be writable by the Apache user? For security reasons I install plugins manually and do not use the extension installer.

Also, we need to make sure that we do not require configure to update content. The TWiki view script need to be able to process old topic formats. I find it a good idea that this stuff is handled in TWiki::Compatibility.

-- PeterThoeny - 21 Jun 2007

It is going to be difficult to remove the upgrade script from the distribution.

Because I already removed it I also removed it from the upgrade guide.

The script is in SVN but it is not in the MANIFEST and therefore not in the distribution package. I was the one that removed it and updated the docs back in January 2007.

I do not think the reminder of this proposal needs to go through the release process as this is code refactorings.

But I fully agree that the configure script should be the tool to perform updates like the recent one where the working area and temp dirs where moved.

-- KennethLavrsen - 21 Jun 2007

Maybe "ditch" doesn't translate well, but it's synonymous with "remove". No disrespect to MartinGregory's work intended.

Kenneth, interesting that you removed it without a topic in Codev..... Bugs:Item3394 for anyone who is interested. I re-opened Bugs:Item3394 to remove it from subversion.

Crawford, I remember that you did not like the idea of using configure for anything else than pure configuration, what is the reason to change your view? - pragmatism. IMHO the upgrade script is the right place for this sort of thing, but it's clear it's never going to be maintained. It's really difficult to do this sort of step in configure - it wasn't designed for doing upgrades - but it seems that it's the only approach that will work. At least we can isolate most upgrade code in the checkers, which helps somewhat. I am very anti the idea of "on the fly" upgrades in the core code. Having just achieved a 16% performance improvement in the core just through selective compilation, I think I have amply demonstrated the reason!

This has also security implications. Does that mean that all libs need to be writable by the Apache user? For security reasons I install plugins manually and do not use the extension installer. - no, it doesn't mean that. The upgrade affects data directories only.

IMHO your security argument just doesn't hold water. TWiki is used in three modes; personal, firewalled and public.

• In personal mode, it doesn't make a hoot of difference for 99.99% of people whether you unzip and copy, or use the extensions installer. Most people don't even bother checking the MD5s, and certainly don't have the knowledge to check the code for security risks.
• In firewalled mode, the risk of hacking is very low. However no admin with a grain of sense is going to use the extension installer directly into a published TWiki; aside from the security risk, you have to test before rolling any extension out to an active user community.
  • My paranoia makes me maintain a parallel private TWiki to the published one, and perform all upgrades in that before transferring the result to the published site. The extension installer works really well for that, as it lets me rapidly try out different plugin combinations. Again, most admins don't have the knowledge to be assess the security risks anyway.
• The same applies for public mode, except that the hacking risk is much higher, so more care is required.

All this is just common sense, and I don't think it's up to us to nanny people into working any other way. Ignoring the extensions installer just means it doesn't get the testing it needs for it to be really robust

It would be much better if we could put some effort into thinking about and documenting the different admin approaches (such as the one I described above) rather than just ignoring the extension installer.

-- CrawfordCurrie - 22 Jun 2007

It was a release meeting decision to remove the script. And the reason it was accepted based on a bug item and not a codev topic was that the upgrade script did not work, had never worked properly and noone would maintain it. We had so many support questions where people had tried the script and goofed up the whole thing. It was also decided to keep it in the SVN in case someone would take over maintenance.

But we know now that noone will and the configure path has proven to be a better and much more customer friendly solution. So I agree that we remove this broken script completely.

-- KennethLavrsen - 22 Jun 2007

After all the work that I and Martin did to create it, its a shame that we have not found a way to utilise this functionality.

I think saying that it never worked it is misrepresenting it a bit - it certainly worked for the corporate installations I migrated with it, and similarly I know of others who did manage to use it productivly.

I am not to suggesting that removing it was bad - just that we need to be more creative in this respect.

One idea is that we bring together all the import plugins into one 'migration tool', and that importing old TWiki installations is part of this functionality.

-- SvenDowideit - 11 Jan 2008