1. More robust tag processing
2. Language Constructs for Templates
   • Looping
   • Descion processing
3. Lookup table tags
   • It would be nice to have ability fill tables on the fly
   • Fill dropdowns on forms.

There are many more and I will add to the list as I remember them.

I believe that better modularization of the code is needed to accomplish a flexiable system that is easy to maintain.

I have started by rewritting some code in wiki.pm I have made an improvement to handleCommonTags by allowing includes to be processed as many times as needed instead of twice. By doing that I also provide the ability to use WikiVariables inside the include string. This allows me to have % INCLUDE{"%TABLEWEBNAME%/TWikiWebsTable.txt"}% where "TABLEWEBNAME = Main" in a web or subwebs preferences are defined. This allows the removeal of some hard code links.

I also seperated and rewrote some of the code in handleCommonTags with the addition of 3 new functions.

1. processTags
2. processPrefTags
3. processBuiltinTags

Below is the new code that I wrote. One of my concerns is that the code may increase a pages processing time due to the fact of making sure we have not orphaned any tags that should be filled.

NOTE The code below will not work if you do not fix the space between %. I will be looking into a do not process tags in this section tag since "PRE" tag is only good for rendering and replacement. I will attach my latested wiki.pm.

[I have replaced the space with <nop>, this way you can copy the code from the page -- AndreaSterbini]

# =========================
sub processPrefTags
{
    # This will replace all of the pref tag variables in the text provided.
    # It then returns the processed text to the caller.
    # Purpose: to allow better processing of pref tags.
    # Added by HaroldGottschalk
    my( $text ) = @_;

    my $x;
    my $cmd;
    for( $x = 0; $x < @prefsKeys; $x++ ) {
        $cmd = "\$text =~ s/%$prefsKeys[$x]%/&handlePrefsValue($x)/geo;";
        eval( $cmd );
    }

    return $text;
}

# =========================
sub processBuiltinTags
{
    # This will replace all of the builtin tag variables in the text provided.
    # It then returns the processed text to the caller.
    # Purpose: to allow better tag processing and accomodate future enhancements
    #    of templates.
    # Added by HaroldGottschalk
    my( $text ,$topic) = @_;

    $text =~ s/%HTTP_HOST%/&handleEnvVariable('HTTP_HOST')/geo;
    $text =~ s/%REMOTE_ADDR%/&handleEnvVariable('REMOTE_ADDR')/geo;
    $text =~ s/%REMOTE_PORT%/&handleEnvVariable('REMOTE_PORT')/geo;
    $text =~ s/%REMOTE_USER%/&handleEnvVariable('REMOTE_USER')/geo;
    $text =~ s/%TOPIC%/$topic/go;
    $text =~ s/%WEB%/$webName/go;
    $text =~ s/%WIKIHOMEURL%/$wikiHomeUrl/go;
    $text =~ s/%SCRIPTURL%/$urlHost$scriptUrlPath/go;
    $text =~ s/%SCRIPTURLPATH%/$scriptUrlPath/go;
    $text =~ s/%SCRIPTSUFFIX%/$scriptSuffix/go;
    $text =~ s/%PUBURL%/$urlHost$pubUrlPath/go;
    $text =~ s/%PUBURLPATH%/$pubUrlPath/go;
    $text =~ s/%ATTACHURL%/$urlHost$pubUrlPath\/$webName\/$topic/go;
    $text =~ s/%ATTACHURLPATH%/$pubUrlPath\/$webName\/$topic/go;
    $text =~ s/%DATE%/&getLocaldate()/geo;
    $text =~ s/%WIKIVERSION%/$wikiversion/go;
    $text =~ s/%USERNAME%/$userName/go;
    $text =~ s/%WIKIUSERNAME%/$wikiUserName/go;
    $text =~ s/%WIKITOOLNAME%/$wikiToolName/go;
    $text =~ s/%MAINWEB%/$mainWebname/go;
    $text =~ s/%HOMETOPIC%/$mainTopicname/go;
    $text =~ s/%WIKIUSERSTOPIC%/$wikiUsersTopicname/go;
    $text =~ s/%WIKIPREFSTOPIC%/$wikiPrefsTopicname/go;
    $text =~ s/%WEBPREFSTOPIC%/$webPrefsTopicname/go;
    $text =~ s/%NOTIFYTOPIC%/$notifyTopicname/go;
    $text =~ s/%STATISTICSTOPIC%/$statisticsTopicname/go;
    $text =~ s/%SEARCH{(.*?)}%/&handleSearchWeb($1)/geo;

    return $text;
}

# =========================
sub processTags
{
    # This will replace all of the tag variables in the text provided.
    # It then returns the processed text to the caller.
    # Purpose: to allow focused processing of tags
    # Added by HaroldGottschalk
    my( $text, $topic) = @_;

    $text = processPrefTags( $text );
    $text = processBuiltinTags( $text, $topic );

    return $text;
}

# =========================
sub handleCommonTags
{
    my( $text, $topic ) = @_;

    # process prefs and multiple embeded 
    Warning: Can't find topic ""."" 
 statements     # added HaroldGottschalk
    $text = processTags( $text ,$topic);
    while ($text =~ /%INCLUDE{(.*?)}%/) {
       $text =~ s/%INCLUDE{(.*?)}%/&handleIncludeFile($1)/geo;  
       $text = processTags( $text, $topic );
    }                        # end add HaroldGottschalk

    # Wiki extended rules
    $text = extendHandleCommonTags( $text, $topic );
    # I do not belive it is needed to process tags again, because the %INCLUDE
    # rules should not fire in extendHandleCommonTags, but if someone thinks it needs
    # to be run again uncomment the line below.  HaroldGottschalk
    #$text = processTags( $text, $topic );

    return $text;
}

-- HaroldGottschalk - 19 Jul 2000

I've been thinking replacing the "=~ s///" code with something similar to the following:

http://prometheus.frii.com/~gnat/yapc/2000-stages/slide48.html and http://prometheus.frii.com/~gnat/yapc/2000-stages/slide49.html

The two advantages of this are: a) it looks nicer, b) you can dynamically register rendering events on a per Web basis.

The only issue I can think of is if $c = eval "sub { /$re/ }" runs slower for s/// expressions.

-- NicholasLee - 19 Jul 2000

I like the code above a lot ... I was just going to change the order of replacements myself ... I want to set a variable with an included file and use variables in the file included ...

P.S. How would you check that there is no mutual recursive file inclusion? Was this the reason for limiting the number of inclusion passes to 2?

-- AndreaSterbini - 26 Aug 2000

AndreaSterbini you bring up a good point that the code does not take into consideration mutual recursion. I do not know the reason for only allowing Includes twice. I reworte that part so tags could get reprocessed, but this does leave a potential hole for a runaway script.

I will think about how we can solve this issue.

-- HaroldGottschalk - 27 Aug 2000

Limiting includes to two levels was an easy solution to avoid infinite loops. We could lift this limitation with a more intelligent algorithm.

-- PeterThoeny - 29 Aug 2000

-- FrancoBagnoli - 01 Sep 2000

Here is an updated version of your code that receives the $theWeb parameter (I've taken the code from current Twiki's beta) . I have also added a preference processing step before processing internal tags so that:

• I can use shortcuts for common strings in templates and topics (e.g. VIEWURL=%SCRIPTURLPATH%/view%SCRIPTSUFFIX% )
• I can use variables inside the arguments of INCLUDE, SEARCH, TOC tools (e.g. %INCLUDE{"%TWIKIWEB%.TWikiWebTable"}%

this is TERRIBLY useful!!!

TODO:

• stop the runaway script problem

Please, Harold, re-edit this topic at your will.

#HaroldGottschalk and AndreaSterbini
# =========================
sub processPrefTags
{
    # This will replace all of the pref tag variables in the text provided.
    # It then returns the processed text to the caller.
    # Purpose: to allow better processing of pref tags.
    # Added by HaroldGottschalk
    my( $text ) = @_;

    my $x;
    my $cmd;
    for( $x = 0; $x < @prefsKeys; $x++ ) {
        $cmd = "\$text =~ s/%$prefsKeys[$x]%/&handlePrefsValue($x)/geo;";
        eval( $cmd );
    }

    return $text;
}

# =========================
sub processBuiltinTags
{
    # This will replace all of the builtin tag variables in the text provided.
    # It then returns the processed text to the caller.
    # Purpose: to allow better tag processing and accomodate future enhancements
    #    of templates.
    # Added by HaroldGottschalk, updated by AndreaSterbini
    my( $text ,$topic, $theWeb) = @_;

    $text =~ s/%HTTP_HOST%/&handleEnvVariable('HTTP_HOST')/geo;
    $text =~ s/%REMOTE_ADDR%/&handleEnvVariable('REMOTE_ADDR')/geo;
    $text =~ s/%REMOTE_PORT%/&handleEnvVariable('REMOTE_PORT')/geo;
    $text =~ s/%REMOTE_USER%/&handleEnvVariable('REMOTE_USER')/geo;
    $text =~ s/%TOPIC%/$topic/go;
    $text =~ s/%WEB%/$theWeb/go;
    $text =~ s/%WIKIHOMEURL%/$wikiHomeUrl/go;
    $text =~ s/%SCRIPTURL%/$urlHost$scriptUrlPath/go;
    $text =~ s/%SCRIPTURLPATH%/$scriptUrlPath/go;
    $text =~ s/%SCRIPTSUFFIX%/$scriptSuffix/go;
    $text =~ s/%PUBURL%/$urlHost$pubUrlPath/go;
    $text =~ s/%PUBURLPATH%/$pubUrlPath/go;
    $text =~ s/%ATTACHURL%/$urlHost$pubUrlPath\/$theWeb\/$topic/go;
    $text =~ s/%ATTACHURLPATH%/$pubUrlPath\/$theWeb\/$topic/go;
    $text =~ s/%DATE%/&getLocaldate()/geo; # depreciated
    $text =~ s/%GMTIME%/&handleTime("","gmtime")/geo;
    $text =~ s/%GMTIME{(.*?)}%/&handleTime($1,"gmtime")/geo;
    $text =~ s/%SERVERTIME%/&handleTime("","servertime")/geo;
    $text =~ s/%SERVERTIME{(.*?)}%/&handleTime($1,"servertime")/geo;
    $text =~ s/%WIKIVERSION%/$wikiversion/go;
    $text =~ s/%USERNAME%/$userName/go;
    $text =~ s/%WIKIUSERNAME%/$wikiUserName/go;
    $text =~ s/%WIKITOOLNAME%/$wikiToolName/go;
    $text =~ s/%MAINWEB%/$mainWebname/go;
    $text =~ s/%TWIKIWEB%/$twikiWebname/go;
    $text =~ s/%HOMETOPIC%/$mainTopicname/go;
    $text =~ s/%WIKIUSERSTOPIC%/$wikiUsersTopicname/go;
    $text =~ s/%WIKIPREFSTOPIC%/$wikiPrefsTopicname/go;
    $text =~ s/%WEBPREFSTOPIC%/$webPrefsTopicname/go;
    $text =~ s/%NOTIFYTOPIC%/$notifyTopicname/go;
    $text =~ s/%STATISTICSTOPIC%/$statisticsTopicname/go;
    $text =~ s/%SEARCH{(.*?)}%/&handleSearchWeb($1)/geo;

    return $text;
}

# =========================
sub processTags
{
    # This will replace all of the tag variables in the text provided.
    # It then returns the processed text to the caller.
    # Purpose: to allow focused processing of tags
    # Added by HaroldGottschalk
    my( $text ,$topic, $theWeb) = @_;

    $text = processPrefTags( $text );
    $text = processBuiltinTags( $text, $topic, $theWeb );

    return $text;
}

# =========================
sub handleCommonTags
{
    my( $text, $topic, $theWeb ) = @_;

    # PTh 22 Jul 2000: added $theWeb for correct handling of 
    Warning: Can't find topic ""."" 
, 
    if( !$theWeb ) {
        $theWeb = $webName;
    }

#AS
    $text = processTags( $text, $topic, $theWeb );
#AS
    # process prefs and multiple embeded %INCLUDE% statements     # added HaroldGottschalk
    $text = processTags( $text ,$topic, $theWeb);
    while ($text =~ /%INCLUDE{(.*?)}%/) {
       $text =~ s/%INCLUDE{(.*?)}%/&handleIncludeFile($1)/geo;  
       $text = processTags( $text, $topic, $theWeb );
    }                        # end add HaroldGottschalk

    # Wiki extended rules
    $text = extendHandleCommonTags( $text, $topic, $theWeb );
    # I do not belive it is needed to process tags again, because the %INCLUDE
    # rules should not fire in extendHandleCommonTags, but if someone thinks it needs
    # to be run again uncomment the line below.  HaroldGottschalk
    #$text = processTags( $text, $topic, $theWeb );

    return $text;
}
# HaroldGottshalk and AndreaSterbini

-- AndreaSterbini - 31 Aug 2000

The problem of the runaway script can be solved using the approach of C headers:

#ifndef thisfile
#include thisfile.h
#define thisfile
#endif

-- FrancoBagnoli - 01 Sep 2000

Probably it's enough to remember the files included so far and pass them to the next call of the handleIncludeFiles routine ... (and stop the inclusion accordingly).

-- AndreaSterbini - 01 Sep 2000

Thanks for the contribution! We can take it into the TWiki core once ironed out. I set the classification to "to do".

-- PeterThoeny - 03 Sep 2000

A couple of ideas about variables/preferences

• What if we add a way to acces variables of a given web ? (e.g. its WEBBGCOLOR). Something like these two ...

• • %VAR{"WEBBGCOLOR" web="Main"}%
  • %Main.WEBBGCOLOR%

• What if all internal tags of TWiki are simply inserted in the SITE var list
  • they are protected from misuse
  • the handler is simpler

-- AndreaSterbini - 12 Sep 2000

I added most of the above code to TWikiAlphaRelease. The preferences handling has been externalized to a new file wikiprefs.pm. Call by reference has been used for performance reasons.

Question to Andrea: processTags is called twice before the first INCLUDE, why is that? (I have currently only one commited)

I extended the getPreferencesValue function in wikiprefs.pm, it accepts now also a $theWeb parameter. This is in preparation of %VAR{"WEBBGCOLOR" web="Main"}%.

-- PeterThoeny - 28 Sep 2000

You are right, only the one inside the loop is needed.

Anyway, I have implemented the recursive INCLUDE (without the runaway script problem ) but it shows a <insert your preferred bad word here> interaction (read "bug") with the implementation of TOC over included files ... as soon I remove the bug I upload the code.

-- AndreaSterbini - 29 Sep 2000

It would be useful to have a uniform "api" to "overload" the access to some variables (set and read) with perl routines, so that for instance a variable can be connected to an entry in a database (say intranet names, passwords, permissions). The "reading" part is trivial, but the "set" part is not: at present the interface to a function is through the %VARIABLE{}% syntax

-- FrancoBagnoli - 30 Sep 2000

Why not using HTML::Template? see also NewTemplateScheme

-- FrancoBagnoli - 10 Oct 2000

I set the classification to FeatureDone since all changes of Harold and Andrea have been implemented and are part of the 01 Dec 2000 production release. Start a new topic for new ideas are enhancment requests.

-- PeterThoeny - 26 Nov 2000