Personal Roadmap for GilmarSantosJr

TWiki Standalone project

I started it as my undergraduation conclusion project. My goal is to enhance performance and decrease resource consumption. I also have some more things in mind:

• A prefs mechanism that uses much less RAM, that can be shared among processes and doesn't get preferences from topics for every request (it would store prefs in something like a dbm database and would get updated every time a topic is saved)
• Some similar enhancement to Forms and Users
• Singleton and classes relationships rethink: simplify architecture and get the chance to take advantage of persistent execution

Performance Improvements

There are many discussions about performance vs compatibility and I think that compatibility is very important, but there must be a limit. I think about some changes that would raise performance a lot, but would break some things. The most important are:

• Build some kind of LALR parser for TML (I know that's not simple, but I'll try sometime as a conception proof) and use it at save time. At view time get the tree and process dynamic information to build final HTML.
• Use retrieval information techniques to improve SEARCH. Query SEARCH will be very fast with DatabaseStore and all other types (except regular expresion, that wil be replaced by Query search in most cases) could be very fast if we use Xapian or some other indexer...
• Performance improvements must be measured somehow, so I want to play with BenchmarkFramework
• TWikiCache also plays an important role

I hope to analyze each one of these and develop as conception proofs, making drop-in replacements whenever possible. After we could figure out together if performance improvements are good enough to think about breaking compatibility (although I call this evolution. I think it's much better for plugin writers to deal with DOM-like trees than with plain text & regular expressions).

Architectural refactoring

Now that TWikiStandAlone is merged and it lets TWiki to run persistently, some changes can be made to raise performance, like:

• Rethink about singletons: separate request-specific from all-runtime-lived singletons
• Rethink about class relationships: what classes really need to be related with each other? (this will help a lot to reduce memory leaks, since it would simplify references)
• Separate request-specific from common-to-all-requests tasks and perform common tasks only once

Other Stuffs

• It would be nice to run twiki inside a jail.I want to achieve this somehow (HTTP engine will help a lot)
• It also would be nice to have TWikiOnMemoryStick using HTTP engine, so it would require only perl to run (I mean, it could run on different OS without changes!)

My Proposals

My action plan

1. Merge TWikiStandAlone into core (Bugs:Item4662)
   1. Create unit tests to the new code (Bugs:Item5846)
   2. Update documentation (Bugs:Item4662#Documentation_updates)
2. Release ModPerlEngineContrib, FastCGIEngineContrib and HTTPEngineContrib
   • Bugs:Item6042
   • I'll try to drop HTTP::Body dependency from ModPerl engine
   • I'll rewrite HTTP engine from scratch, based on CPAN:Net::Server, that is more powerful and flexible than CPAN:HTTP::Daemon
     • Add support to run configure script from HTTP engine
3. Develop a TWiki::Prefs that uses less RAM and that is faster (no need to parse preferences for each request)
4. Develop some kind of BenchmarkFramework
5. Work on architectural refactoring
   1. Produce high level documentation (Diagrams, Models, Specifications)
   2. Make it possible to use shared memory
   3. Perform some tasks only once when persistent engines are in use
6. Study about unit test coverage using Devel::Cover
7. Play with some kind of LALR parser for TML

-- GilmarSantosJr - 28 Feb 2008

Discussion