Performance Bottlenecks

One of the key challenges facing the developers is speeding up TWiki, especially with large numbers of topics. These include:

SEARCHing topic text

Problem

Every topic that is in the search scope must have all of its tags expanded before the search. According to DakarPreferenceModel, this means up to 17 places are searched for every tag. In addition, commonTagsHandler is called for each plugin that implements it.

I have to assume that this is slow and, obviously, proportionately slower for every plugin that implements commonTagsHandler

-- Contributors: MeredithLesly

Discussion

I am certainly not one of the developers most familiar with the code, so I hope that those who are more so will correct any errors I've made. And, more importantly, come up with solutions or

-- MeredithLesly - 22 May 2006

Can you give an example of the above (expanding the tags on search)? I believe only raw text is searched (at least in Dakar code base), unless we are talking search plugins.

-- SteffenPoulsen - 22 May 2006

The preference lookup is not "slow" thanks to the caching. That means that the "cost" (percentage of the total time) of reading the preferences is lower as the number of topics increases.

-- RafaelAlvarez - 22 May 2006

Which preferences are cached? Every topic's preferences?

SP, you're partially right: expandvariables is an option to search, so only searches that have that set will expand the tags. This means, of course, that if = expandvariables= isn't set, topics that the user expects to found aren't. This is something that is likely to elude many developers (including me until now!).

-- MeredithLesly - 22 May 2006