Question

Is there any interest in speeding up TWiki by creatively using caching and possibly FastCGI?

-- JoelRegen - 14 May 2004

Answer

Actually, TWiki can be reasonably fast. The speed depends on the hardware used and on the system load. TWiki.org is currently running on SourceForge, they have a high load on their web servers, cgi scripts are throttled sometimes to a crawl. Once we move TWiki.org to a different server you will see better performance.

Some people run TWiki under ModPerl, FastCGI, SpeedyCGI.

Caching for fast access is available, see TWikiCacheAddOn, CacheAddOn, PublishAddOn.

-- PeterThoeny - 15 May 2004

I can add my own experiences. We are currently running a large-ish TWiki site, two Sun V100 "appliance" servers running Apache, both sitting behind an F5 loadbalancer. The servers run multiple virtual servers. We slowly brought each TWiki cgi under SpeedyCGI and saw a drastic improvement in responsiveness. It is my belief that the lowly V100's simply do not have the memory bandwidth needed to handle the multiple tasks being requested - keeping Perl in memory definitely made a big speed improvement and we have yet to run into any persistence issues. I will say that our first test was using the Apache module and that sunk like a lead balloon - EditTablePlugin croaked. Modifying each CGI did the trick AND it provides for more finer control and memory partitioning (provided for by SpeedyCGI) if required.

-- SteveRJones - 02 Jun 2004

Reply

Ok, I read some of the postings under ModPerl and ModPerlUnix and my first impression is to run away like a Monty Python retreat. I mean, come on...there has to be a better way to deploy software. No? I tried Koala Skin and it broke TWiki so I got a bad taste in my mouth from that. Now this nightmare of problems with ModPerl that seem to be caused by too many combinations of versions of modules and applications. Sheeeeeesh!!!! Ok, rant over. Now, how do we solve this problem? I think the first thing is to step back from it and evaluate what we are really doing. (We being software developers). We try to create programs that run within certain environments in order to provide useful functionality for people and hopefully improve their lives and their businesses. But we get caught in a huge morass of complexity due to the increasingly dynamic rate of change of the modules we build with and the features we attempt to add. So, how do we stop this problem? By developing a way to deploy software in a more controlled manner that checks for dependencies. And by using techniques and technologies that lend themselves to higher quality software products. Could it be that Perl is being stretched beyond its capabilities by the complexity of the TWiki feature set? Is there a better language or technology? I don't know. Sorry. Carry on.

-- JoelRegen - 15 May 2004

I couldn't agree with you more, Joel.

This is the thinking behind ExplorationThenConsolidation - that a key responsibility of the CoreTeam has to be to ensure stability in the architecture. A good place to debate these issues is on Codev.TWikiIrc - I'm happy to discuss subjects like the ArchitectureDiagram and CodevDocumentationProject. CrawfordCurrie (aka CDot) is a good person to speak with about SharedCode and Unit Testing; if you agree that Crawford is doing a good job you might want to support his nomination onto the core team. (CoreTeamNominationDiscussions)

-- MartinCleaver - 16 May 2004