Idea: Render a Topic to HTML after each edit and store it
The behaviour of TWiki rendering each page to HTML whenever the Topic is requested ever and ever again produces a high load on web servers.
The idea is to generate the HTML page only once after editing the Topic and store it in a HTML file. Later view requests only get the HTML file displayed directly.
Advantages: 
, which uses cache. It might make sense if you have 1 edit for 100 views. For TWiki (as explained above) it is not as important, we hope to have more edits 
-- PeterMasiar - 25 Apr 2002
This is somewhat like calculating the Last-Modified time for a rendered page (so that proxy caches can work better) - you have to look at every variable in a page, including embedded searches such as FormattedSearch, and then calculate the last-modified times of the various 'input pages' (e.g. a page that is included in a search embedded in another page), and take the most recent time.
In the suggested pre-rendering scheme, you would have to do this in reverse - each time a page (i.e. an 'input page') was created, you would have to re-run all page rendering done in the TWiki site, as John points out, just to find the input page's WikiWord in perhaps thousands of pages, in order to update the rendered (cached) versions of those pages that include a search that returns this 'input page'.
Trying to make this work efficiently would involve a lot of indexing (e.g. a keyword index of every word in the TWiki site) and would probably require that TWiki uses a relational database just to keep track of which embedded search pages should be updated when an input page changes. While this might be good for searching, this wouldn't address the issue of regular expression searches, which can match multiple words and are virtually impossible to build keyword indexes for.
I don't think this enormous increase in complexity, and heavy loading on saves, is worth the improvement in performance on views. If anyone has a site that is so heavily loaded that TWiki view performance is an issue, they should first install SpeedyCGI (similar to ModPerl, but doesn't require webmaster privileges) to see how that improves things. ModPerl gives a roughly ten times speedup, and I expect SpeedyCGI would not be far off that.
I really can't see a way round the complexity issue, and I think SpeedyCGI is a much more elegant way of improving performance. Putting in pre-rendering has a surprisingly large impact on the way TWiki works.
-- RichardDonkin - 25 Apr 2002
- Much less load on the web server (and more concurrent users possible)
- Faster response time
- ...
- Extra effort to fill dynamic parts of the Topic (i.e. results, ...)
- Extra number of files on disk (extra space required)
- ...
- After editing a topic:
- Render only the Topics - not the skins around them. This results in stripped down html files (no <head>, no <body> tags, ...). Save them together with the topic itself.
- Do not interpret the variables (keep them specifically tagged in the resulting html file).
- When a view request is encountered:
- The skin of a topic is always be handled the actual way.
- Copy the html piece to the requester. Only when encountering the prevoiusly tagged variables additional work have to be done.
, which uses cache. It might make sense if you have 1 edit for 100 views. For TWiki (as explained above) it is not as important, we hope to have more edits
-- PeterMasiar - 25 Apr 2002
This is somewhat like calculating the Last-Modified time for a rendered page (so that proxy caches can work better) - you have to look at every variable in a page, including embedded searches such as FormattedSearch, and then calculate the last-modified times of the various 'input pages' (e.g. a page that is included in a search embedded in another page), and take the most recent time.
In the suggested pre-rendering scheme, you would have to do this in reverse - each time a page (i.e. an 'input page') was created, you would have to re-run all page rendering done in the TWiki site, as John points out, just to find the input page's WikiWord in perhaps thousands of pages, in order to update the rendered (cached) versions of those pages that include a search that returns this 'input page'.
Trying to make this work efficiently would involve a lot of indexing (e.g. a keyword index of every word in the TWiki site) and would probably require that TWiki uses a relational database just to keep track of which embedded search pages should be updated when an input page changes. While this might be good for searching, this wouldn't address the issue of regular expression searches, which can match multiple words and are virtually impossible to build keyword indexes for.
I don't think this enormous increase in complexity, and heavy loading on saves, is worth the improvement in performance on views. If anyone has a site that is so heavily loaded that TWiki view performance is an issue, they should first install SpeedyCGI (similar to ModPerl, but doesn't require webmaster privileges) to see how that improves things. ModPerl gives a roughly ten times speedup, and I expect SpeedyCGI would not be far off that.
I really can't see a way round the complexity issue, and I think SpeedyCGI is a much more elegant way of improving performance. Putting in pre-rendering has a surprisingly large impact on the way TWiki works.
-- RichardDonkin - 25 Apr 2002 
Topic revision: r6 - 2002-04-25 - RichardDonkin
Site map
Blog web
Create New Topic
Index
Search
Advanced search
My topics of interest
Changes
Major changes only
All tags
Notifications
RSS Feed
Statistics
Preferences
Community 
Readme first
Developers News
How you can help
Community roles
#twiki IRC
Lima Release
Feature Proposals
Bugs Changes
Categories 
Account 
Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.Copyright © 1999-2026 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.