Seems like it might be a good idea to have an XML (or other) based "WIF": Wiki Interchange Format. Oh, wait! There already is one! It's called HTML or XHTML - all one need do is take a rendered-to-markup wiki page and reverse it back into one's own wiki dialect. Zounds! one could even use it to import new/interesting content into one's wiki! Another sticky wiki solved (with a nod and a wink to Martin Cleaver ).

Come to think of it, one could actually store wiki pages in HTML/XML and only convert them to editable form when the (i'm assuming relatively rare) occassion arises when someone wants to edit a page - think of the reduction in server overhead (alternatively, dual representation?)!

-- DavidLeBlanc - 20 Jun 2001

Interesting idea. Of course it might be better to store it in some intermediary language which is faster to parse. Ties in somewhat with some ideas I've had about Template-Toolkit.

See also: CanonicalTopicStoredForm

-- NicholasLee - 20 Jun 2001

(N.B. I moved the "See also" above Nicholas' signature to give credit for the reference where due.)

Storing a TWiki page in some intermediate format completely defeats the purpose of storing it in HTML/XHTML i.e. that of making view a fast, no-load operation.

Of course, using an XML dtd and CSS or XSLT style sheet would probably be best. Then one might decide to render before sending in the case of older browsers vs. browsers that can handle xml directly. Actually given the varying abilities of browsers with respect to xhtml (I think), it might not be too bad an idea to have a "render on demand" feature that goes to HTML or XHTML based on the "requesting agent" metadata sent with the get.

-- DavidLeBlanc - 21 Jun 2001

I don't think HTML would make a great interchange format. This particularly affects higher level Wiki markup, beyond bold/italic/bullets etc - e.g. an InterWikis link would mean something in several Wikis so should be preserved as Wiki:FrontPage, rather than rendered down into HTML from which it is hard to convert it back to an Interwiki link.

XML is ideal for defining customised languages that capture the semantics required, and is very easy to parse using XML tools (just suck the page into an XML object tree) - it might be better to make the native format an XML-based language (hidden from the user, who would see only Wiki syntax in the edit page), which can then be either exported into other Wikis or easily rendered into various HTML flavours, WML, PDF, and so on.

Decisions about native formats should not be based on speed issues IMO - TWiki is quite fast enough even without ModPerl, I think, and I'd prefer any optimisations to simply cache TWiki ouput rather than affect the choice of native format.

-- RichardDonkin - 21 Jun 2001

Hmmm... isn't TWiki output HTML?

I was thinking about this whole thing further, and realized that TWiki reads some number of files on each page fetch thus further underscoring the utility of this!

To show one topic, it has to fetch the .txt file, the .tpl file, the web preferences and the site preferences, possibly skin files and potentially any other embeded images etc. Given that 10 years ago it took (relatively speaking) 5 decades to fetch a file (comparing cpu time to disk access time), by saving rendered output and only checking the timestamps of the relevent files, one could save a few centuries of computer time on each fetch by caching

The cpu/disk time analogy was that if a single cpu instruction took 1 second, it would take 54 years to fetch something from the disk. This was 10 years ago when cpu seconds where a lot longer

-- DavidLeBlanc - 15 Jul 2001

A couple of points:

• Is TWiki fast enough already?
• Caching is complicated

If TWiki is already fast enough then caching is best avoided. I already face a proxy at work that gives out of date copies of TWiki pages. If caching is to be done in TWiki then the problem is on of links. Any new topic might change the rendering of another, so if caching is to be of benefit, one would need to know which topics are affected by the creation of a new one (otherwise all caching would be deleted every time a topic is created) - a non trivial task.

I think the only part of TWiki that really needs a speed boost is searching. An index of words would help greatly in this, and has significant overlap with the general caching issue.

-- JohnTalintyre - 16 Jul 2001

This depends heavily on the box that is hosting TWiki. My TWiki runs on a relatively slow box, and caching would create a tremendous speedup. (Then maybe getting the number of RCS calls down would help just as much.)

Is there a useful way to profile TWiki?

-- JoachimDurchholz - 17 Jul 2001

You can do some simple profiling by uncommenting the writeDebug call in the writeDebugTimes sub at the top of bin/view (in the latest beta). You'll find output in data/debug.txt

Are you using modPerl? I get okayish performance at home on a 266 running Win2K and Apache/ModPerl - mind you there's no loading.

And yes removing RCS calls does make a fair difference. Latest beta does this (for view) once topic format is upgraded, either on topic edit/save or if the upgrade tool in cvs tools directory is used.

-- JohnTalintyre - 17 Jul 2001

I am not currently running under mod_perl. My concerns wrt to performance are less about today, although, performance on a dual pentium pro II isn't what i'd call sparkling, but rather with a view to the future. Performance is something that needs to be planned for and integrated as you go, not something that's attempted to be bolted on after the fact.

-- DavidLeBlanc - 17 Jul 2001

I've been suprised by the good performance of TWiki and it's made me re-think design somewhat as TWiki manages to keep a lot of things simple, but still give good performance! We found performance unacceptable without modPerl, please give it a go. I used ActiveState perl, I found the upgrade to modPerl very easy using their ppm package manager. If that doesn't improve performance enougth, then you might want faster discs. The only part of TWiki performance that currently concerns me is search performance.

-- JohnTalintyre - 18 Jul 2001