TWiki Refactoring Project

This page is hopefully going to become the central gateway for TWiki refactoring efforts. The TWiki codebase as it is now (2004-02-08) is a mess, and needs some work. To be clear, things linked from this page should be about refactoring. Not feature requests, not bug fixes, but just internal architechture changes to make TWiki easier to work with without affecting it's interface to the world. Now, some of these proposals will have interface consequences...but these are more in the manner of side effects rather than actual goals.

• RefactorCommonTags
• RefactorObjectOriented
• RefactorFatCgi
• RefactorPluginAPI
  • RefactorFuncDotPm
• RefactorOopsTemplates
• PrefsDotPmRefactor
• RenderDotPm
• TWikiDotPmProposedFurtherModularisation

Some minor simple code changes either in Alpha or in the works

• Moved all regex variables into a %regex hash. Added a function getRegularExpression to TWiki::Func to allow plugins to retrieve these, and modified the bundled InterwikiPlugin to take advantage of this. -- WM 09 Feb 2004

---

Initial Introduction

While reading the code and adding documentation on the various functions, I'm going to comment here on architectural peccadillos I encounter that I think it would be possible eliminate without breaking backwards compatibility. I'd like feedback on any of these things; have they been tried and failed? Is there a good reason why things are done the way they are?

Also, many of these refactorings are performance-conscious to various degrees. If anyone has a process for profiling a TWiki request at the function-call level the results could be very interesting. Right now the only thing present is writeDebugTimes, which would require a non-trivial effort to apply to a large portion of the rendering code and then parse the results into some useful form.

-- WalterMundt - 06 Feb 2004

Mentally prepare to delete content!

On TWiki.org almost no content gets deleted! Instead old or irrelevant content gets archived in TopicnameOld, TopicnameArchiv... This behavior was leading to our current situation, where the CodevWeb has a more than 2500 topics which makes an effective work almost impossible.

Content must be held up-to-date as it has to provide useful information for the user. So rather preserve old content in archives it should be refractored and consolidated - damn this is a Wiki!

So keep an eye on topics you visit and

• control and update the Metadata provided in Formfields and
• refractor their content by yourself or
• mark them with an appropriate WikiBag like DeleteMe, RenameMe, RefractorMe etc.

Of course keep in mind WikiRefactoring when refractor topics

-- AndreUlrich - 09 Feb 2004

I've come to realise that the TopicnameOld, TopicnameArchiv... archiving is a symptom of not having either an easy method to link to old revisions (RevRecoverPlugin) or the ability to annotate revisions like wikipedia (try out the tick boxes on that page)

-- SamHasler - 24 Feb 2004

---

Michael and Will have gone a good way into refactoring the code at O'Wiki and continue to offer their changes for the core team to lift. I do hope that you plan to make good use of their work.

-- MartinCleaver - 06 Feb 2004

If they don't feel able to use it for whatever reason, despite the constant offers, I hope you feel able to look at what's been done (slaving TWiki.cfg off a config hash - making > 60 vars redundant, using Variable::Alias to help as a restructuring tool whilst keeping a running, stable system. (No "big bang" approach)

So far well over half are now redundant in our tree and ready for deletion with extreme prejudice, or already have been.

-- MS - 06 Feb 2004

It's a question of an individual developer putting a lot of time into taking code from OWikiFork and putting it into TWiki. I'm not sure there is anyone with that motivation at present. I've got nothing against O'Wiki, but I'd personally prefer to re-factor directly, as I've done in the past with TWiki. Mind you, this is somewhat academic, as I have not time for TWiki developement at present.

I'm really just trying to say that it's no easy matter to take refactored code from O'Wiki into TWiki.

-- JohnTalintyre - 06 Feb 2004

MS, as I had been somewhat detached from the TWiki community until rather recently, I haven't really been tracking your OWikiFork. When I was last involved regularly, you had just decided to create a fork. I think I'll definitely go grab it and look through the code. I don't know whether I'll try and merge changes into TWiki or just use them for inspiration, but I'm sure they'll be helpful. Since I do have some time to spend, I think that it would be a good way to invest it; you're obviously a much more experienced developer than I am, and it will be interesting to see how you've approached the problem. A 'big bang' approach to refactoring things was exactly what I was leery of; committing that kind of change into TWikiAlphaRelease all at once would be asking for trouble. There's no way I could possibly test things well enough here to prevent a hundred problems from cropping up.

-- WalterMundt -- 06 Feb 2004

John - it's an offer, not an expectation. (Indeed my expectation is for my work not to be used, or if it is to be used in a manner that breaks my content). The offer however stands, and if any developer needs CVS access to test their ideas, I'm more than willing to provide it. If they don't do any of those things, that's not a problem from my perspective.

I would be interested to hear how you think the community at large without CVS access can perform large scale refactoring of the code in such a manner that the CoreTeam would be happy to accept - and not bland generalities I could easily just attach a cvs diff -u= to a page, but that wouldn't actually help.

Culling a couple of thousand lines of base TWiki code, and eliminated most of the bin scripts whilst adding large amounts of extra functionality (leaving just stubs) is a pretty large refactor, but your implicaton above is that it can't be accepted. (This isn't a problem, it's why I have my own fork) Without CVS access the TWiki community cannot directly change the TWiki code, especially due to the patch application rate. (again, not a whinge, just a fact of life) The closest anyone can do is to provide diffs. The most effective way I can do this effectively is to allow you CVS access to my code, and allow you to perform diffs along the lines of cvs diff -u -r TWIKI <file>. It makes no demands, just makes offers. (Which don't have to be accepted)

• Incidentally it was this specific issue that made me think of the CVS code tree as wiki (CHAOS) in the first place.

Walter - I've no idea who's more experienced, and IMO that doesn't matter. If you chose to do it your own way because it's more fun that way, or you gain something else from it (like much better skills than the rest of us combined) then great . Sooner or later our codebases will drift as a result, but again, I don't view that as a problem. I'm always happy to throw away stuff I've written given sufficient motivation - such as increased clarity, or functionality (like MetaDataRendering, in favour of other things - like FormattedTWikiFormDataInTopicText).

As noted above they're offers, not demands, etc. Much like the "merry christmas, take it, leave it" note at the bottom of named include sections.

-- MS - 09 Feb 2004

I very much agree with what MS and Walter say above. It's great to have OWikiFork as an offer and I think it is more likely to spark (no pun intended) inspiration and taking small code areas rather than large scale introduction back into TWiki. I was trying to give a realistic answer to Martin saying above "I do hope that you plan to make good use of their work.".

MS said he couldn't see how a large scale refactor could work without CVS access - I totally agree. I hope we can get more people into the CoreTeam i.e. having CVS access. I did do lots of refactoring before I had CVS access - it was even more work to bring it into the TWiki code base; lots of diffing and testing, but also more careful thought and discussion of some of the changes I'd made.

-- JohnTalintyre - 09 Feb 2004

Moved discussion of %regex refactor release down here:

CC: Walter, great start - please can we have this released, as soon as humanly possible.
WM: I had already committed this to CVS by the time I wrote the description, which is why it's in past tense. Either do a CVS checkout or wait for the next automated TWikiAlphaRelease to get packaged; it'll be in there.
CC: Don't want an alpha, want a production. I have no way of synching plugins releases with anything but production releases.
WM: Ahh. That might be an issue; I don't know how far off CairoRelease is. Another thing to keep your eye on is RefactorPluginAPI, which might also make it in before CairoRelease. We shall see.

-- WalterMundt - 09 Feb 2004