CLOSED original development topic for Table Of Contents %TOC% variable

• Ongoing TOC dev discussion in TocVariableDev
• Documentation in TWiki.TWikiVariables
• Implementation details in TWiki.TWikiHistory

I propose to introduce a new tag to show an index linked to all SectionTitles inside a topic (i.e. H1..H6 html tags). This could be used at the beginning of the topic to index the section titles that follows. (similarly to a mail digest in the mailing-lists world)

I propose the %TOC{"Web.Topic"}% tag, so that we can show in a topic the index of any topic in any web. (if the topic is not specified it defaults to the current one).

The index should be produced by collecting all SectionTitles, that are specified with the following simple sintax:

   + Heading 1
      + Heading 2
         + Heading 3
               ... and so on  

The index could be indented along some rule (Ibis rules or simpler) so that we could have a simple nice threaded discussion index.

Example, suppose the topic contains:

   + Heading One

Blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
blah blah blah blah 

      + Heading Two

Blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
blah blah blah blah 

         + Heading 3

Blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
blah blah blah blah 

Then %TOC{"Codev.SectionTitles"}% is transformed to the outline

• Heading One
  • Heading Two
    • Heading 3

And the above text is shown as:

Heading One

Blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah

Heading Two

Blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah

Heading 3

Blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah

-- AndreaSterbini - 23 Aug 2000

Here is some code to do the above tricks (in wikicfg.pl):

• Add this line at the end of extendHandleCommonTags

#AS   should be moved to renderOutsidePRE, but needs the $topic parameter
    $text=~ s/%TOC{([^}]*)}%/&handleToc("$theWeb","$topic",$1)/geo;    
#AS

• And add these routines:

#AS
# =========================
sub showError
{
    my( $errormessage ) = @_;
    return "<<nop>FONT SIZE=\"-1\" COLOR=\"#FF0000\">$errormessage<<nop>/FONT>" ;
}

# =========================
sub handleToc
{
    # Andrea Sterbini 22-08-00
    # Routine to create a TOC bulleted list linked to the section headings
    # of a topic. A section heading is entered in one of the following forms:
    #  <tab><tab>...+ section heading
    #  <tab>++... section heading
    #  *Hn: section heading*
    # Parameters:
    # $theWeb   = the web we are in
    # $theTopic = the topic we are in
    # $attributes = "Topic" [web="Web"] [depthLimit="N"]

    my( $theWeb, $theTopic, $attributes ) = @_;
    
    # get the topic attribute
    my $topicname = extractNameValuePair( $attributes );
    if ( ! $topicname ) {
        $topicname = $theTopic;
    }
    
    # get the web attribute
    my $web = extractNameValuePair( $attributes, "web" );
    if ( ! $web ) {
        $web = $theWeb;
    }
    $web =~ s/\//\./go;

    # get the depthLimit attribute
    my $depthLimit = extractNameValuePair( $attributes, "depthLimit" );
    if ( ! $depthLimit ) {
        $depthLimit = 6;
    }
    
    my $list  = "";
    my $line  = "";
    my $level = "";

    my $webPath = $web;
    $webPath =~ s/\./\//go;
    

    my $fileName = "$dataDir/$webPath/$topicname.txt";
    if ( -e $fileName ) {
        my @list = split( /\n/, readFile( $fileName ) );
        @list = grep { /(<<nop>\/?[pP][rR][eE]>)|(^\t+\+\s+.*\s*$)|(^\t\++\s+.*\s*$)|(^\t\*H[1-6]:\s+.*\*\s*$)/ } @list;
   my $insidePre = 0;
   my $i = 0;
   my $tabs = "";
        foreach $line ( @list ) {
       if ($line =~ /^.*<<nop>[pP][rR][eE]>.*$/ ) {
      $insidePre = 1;
      $line = "";
       }
       if ($line =~ /^.*<<nop>\/[pP][rR][eE]>.*$/ ) {
      $insidePre = 0;
      $line = "";
       }
       if (!$insidePre) {
      $level = $line ;
      if ( $line =~  /^(\t+)\+\s+(.+)\s*$/ ) {
          $level =~ s/^(\t+)\+\s+(.+)\s*$/$1/go ;
          $level = length $level ;
          $line  =~ s/^(\t+)\+\s+(.+)\s*$/$2/go ;
      }
      if ( $line =~  /^\t(\++)\s+(.+)\s*$/ ) {
          $level =~ s/^\t(\++)\s+(.+)\s*$/$1/go ;
          $level = length $level ;
          $line  =~ s/^\t(\++)\s+(.+)\s*$/$2/go ;
      }
      if ( $line =~  /^\t\*H([1-6]):\s+(.+)\*\s*$/ ) {
          $level =~ s/^\t\*H([1-6]):\s+(.+)\*\s*$/$1/go ;
          $line  =~ s/^\t\*H([1-6]):\s+(.+)\*\s*$/$2/go ;
      }
                if( ($line) && ($level <= $depthLimit) ) {
              $tabs = "";
          for ($i=0 ; $i<$level ; $i++) {
         $tabs = "\t$tabs";
          }
          $line = "$tabs* <<nop>A HREF=\"$scriptUrlPath/view$scriptSuffix/$webPath/$topicname#$line\">$line<<nop>/A>";
                   $list = "$list\n$line";      
           }
       }
        }
   if ( $list ) {
           return $list;
   } else {
   return showError("No TOC in \"$web.$topicname\".");
   }
    }
    return showError("Cannot find topic \"$web.$topicname\".");    
}
#AS

• Finally, add these lines to extendGetRenderedVersionOutsidePRE

#AS
    #
    # render <tab>(+++++++) text as "<<nop>Hn>" text:
    #
    s/^\t(\++)\s(.*)\s*$/"<<nop>H" . (length $1) . "><<nop>A NAME=\"$2\">$2<<nop>\/A><<nop>\/H" . (length $1) .">"/geo;
    #
    # render (<tab><tab><tab><tab>+)+ text as "<<nop>Hn>" text:
    #
    s/^(\t+)\+\s(.*)\s*$/"<<nop>H" . (length $1) . "><<nop>A NAME=\"$2\">$2<<nop>\/A><<nop>\/H" . (length $1) .">"/geo;
    #
    # render \t*Hn: text* as "<<nop>Hn>" text:
    #
    s/^\t\*H([1-6]):\s+(.+)\*\s*$/<<nop>H$1><<nop>A NAME="$2">$2<<nop>\/A><<nop>\/H$1>/go ;
#AS

• That's it! (remove the <nop> if you are copying the code from the edit form)

The code above does distinguish between <PRE> formatted text and the rest, it collects all the SectionTitles only outside <PRE>.

-- AndreaSterbini - 23 Aug 2000

Proposal for an alternative format

A more compact format for <Hn> tags could be the following (we count the number of +).

   + H1
   ++ H2
   +++ H3
   ++++ H4
   +++++ H5
   ++++++ H6

A rule to render it is already listed above (and handled by the code).

How to handle the <PRE> formatted text

A quick hack to be sure the above rules dont fire in <PRE> tags is to store at the beginning of each line of a <PRE> formatted text a <nop> null tag. (this, suggests the introduction of some sort of CanonicalTopicStoredForm).

I have modified the above code to avoid <PRE> formatted text.

-- AndreaSterbini - 28 Aug 2000

AndreaSterbini, this is an excellent idea! I would like comments from other users regarding the syntax. Previously I have thought of a *H4: San Jose* style syntax for the headings as described in StructuredTextEnhancements. It should be very easy to remember. Comments?

-- PeterThoeny - 29 Aug 2000

I like that format ... I have modified the code to use it also.

The handleToc routine above now uses the parameters:

E.g. you could use the tag this way %TOC{"TocExamples" web="TWiki" depthLimit="3"}%

-- AndreaSterbini - 01 Sep 2000

Looks good, we can soon take it into the TWiki core. For that I prefer to offer only one rule, not all. Which one is more intuitive?

Also, one thing that needs to be considered are includes. What to do if other topics with headers are included into a web where the TOC is being built? Probably need to build the TOC that includes the included headers (wheeuh, what a tongue breaker!). The TWikiDocumentation topic is an example where the topic has a (manually written) TOC and all text included.

Including the includes when building the TOC could be used to write a whole book in TWiki!

I changed the TopicClassification to a "to do" item.

-- PeterThoeny - 03 Sep 2000

I would like to take Andreas's contribution into the TWiki core since it is part of StructuredTextEnhancements. Before that I would like to get opinions on the syntax from other people by a vote. I added yet another syntax. Feel free to do the same. We will choose one (max. two) for the TWiki core.

Proposed syntax for defining headers:

1. "indent indent plus TEXT" type: Similar to bullet list. Example:
      + Sushi
      + Maguro
2. "indent plus plus TEXT" type: More compact then 1. , easier to type. Example:
      + Sushi
   ++ Maguro
3. "star 'H' number colon TEXT star" type: Easy to remember, similar to *bold*. Example:
   *H1: Sushi*
*H2: Maguro*
4. "dash dash star TEXT" type: Visual clue of header when in edit mode, similar to separator. Syntax is 4 or more dashes, then one ore more pluses representing the level. Example:
   -----+ Sushi
-----++ Maguro
5. "dash dash 'H' number colon TEXT" type: Similar to 3. , but with 'H'. Example:
   -----H1: Sushi
-----H2: Maguro
6. (what else?)

Vote on syntax:

Just trying to keep the ballot box full -- see also http://www.c2.com/cgi/wiki?WikiEngineReviewTextFormattingTest -- note that it is neither 100% complete nor 100% correct.

-- RandyKramer - 28 Feb 2001

• [ PeterThoeny ] Note for Randy: I have looked at the page you mentioned. Can't use '=' because already in use.

• [ RandyKramer ] Peter: Thanks! I noticed that after I read this page. After you read this, feel free to delete this note.

Additional idea: We could extended section titles with automatic numbered headers. Example text entered:

=-----N1: Some title=
bla bla
=-----N1: Next title=
bla bla

This would get rendered as

<h1>1. Some title</h1>
bla bla
<h1>2. Next title</h1>
bla bla

-- PeterThoeny - 12 Sep 2000

I would prefer that we do not reuse characters already in use. I propose using the # (sharp, pound, octothorpe, spider, whateveryouwanttocallit) in part because it looks something like an H. Thus my favorite syntax above becomes:

# Sushi
## Maguro

As for concerns regarding visibility when editing summarized in items 4 and 5 above, headers are already likely to be surrounded by vertical white space and thus are relatively easy to locate. A bigger problem in my view is the way that tables are edited. See TableEditingProblems for further discussion.

-- JohnAltstadt - 13 Sep 2000

My commentary is rather simple. Embedding H2 H1 in wiki may make sense to those familiar with HTML, but it will not make sense to those who are not.

More crucually, it does not scan as the intended meaning. You cannot see *H2: Fruitbats* as a Fruitbats section without explictly parsing out H2, considering its meaning and applying that to its neighbor. This seems contrary to the goals of WikiMarkup. For this reason, I favor implementations which are more visually indicitave of their meaning.

-- JoshuaRodman - 14 Sep 2000

I agree that the rule should be easy to remember. Granted, *H2: Something* is kind of HTMLish, not necessarily easy to remember for people not knowing HTML. I kind of also like 2. and 4. , we could go for that. Nevertheless, we should think ahead about the syntax for numbered titles as well; the rule should be easy to remember for unnumbered and numbered titles.

FWIW, Here is how SDF (a simple markup language for creating large docs that need to be translated into different formats) defines titles:

So, H1 ... H6 define numbered titles, P1 ... P6 define unnumbered titles.

BudBruegger changed the syntax of his experimental TWiki clone to support the SDF syntax, see an example topic at http://www.sistema.it/twiki/bin/view/Main/TestArea .

Any ideas, opinions?

-- PeterThoeny - 16 Sep 2000

Incorporated Andrea's TOC and heading plugin into the TWikiAlphaRelease. That means, TocPlugin is no more needed. See spec for TOC in TWikiVariables, spec for headers in TextFormattingRules.

TOC:

• Write: %TOC%
  • You get the table of content of the current topic, including all included topics. TOC is a indented list of linked heading text.
• Write: %TOC{"TopicName" web="Webname"}%
  • You get the TOC of the specified web. Note that included topics are ignored at the moment.
• Write: %TOC{depth="2"}%
  • You get the TOC of the current topic with the headings limited to level 2.

Headings:

There are three format that are understood. Wondering which one to document "officially".

1. "---++ text" format:
   Three or more dashes at the beginning of the line, pluses (one plus is level one...), heading text.
2. "   ++ text" format:
   Three spaces at the beginning of the line, pluses (one plus is level one...), heading text.
3. "<H2> text </H2>" format:
   The HTML way; the opening tag needs to be at the beginning of the line. (TWiki will expand it with an achor)

If interested, here is the code for creating the #anchor name out of the heading name:

    $anchorName =~ s/^\s*\#?//o;          # no leading space nor '#'
    $anchorName =~ s/\s*$//o;             # no trailing space
    $anchorName =~ s/[^a-zA-Z0-9]/_/go;   # only allowed chars
    $anchorName =~ s/__+/_/go;            # remove excessive _
    $anchorName =~ s/^(.{32})(.*)$/$1/o;  # limit to 32 chars

The next planned step is AutomaticallyNumberedHeadings.

-- PeterThoeny - 28 Feb 2001

Enhanced code so that the TOC is built correctly of topics that include other topics. Commited to TWikiAlphaRelease.

-- PeterThoeny - 04 Mar 2001

I'd like to patch my December release to include the latest version of this feature. Anyone got a diff that I could use?

-- MartinCleaver - 09 May 2001

Your best bet is to upgrade to the latest Beta, 15 Mar 2001.

-- PeterThoeny - 11 May 2001

Great minds think alike, and I don't always keep up to date with what other people are doing (oops). We have a home-grown plug-in that our technical authoring team wanted, that supports the following:

• Section headings and AutomaticallyNumberedHeadings, as in Andrea's plug-in
• ....numbering works across topics, using an ordering deifned in a "WebOrder" topic
• %TOC% for the full web supported, with %TOC{topic="Topic"}% as well.
• Symbolic anchors and cross-references embedded in text %ANCHOR{type="Figure", name="symbolicName"}% and %REF{topic="OtherTopic",type="Figure",name="symbolicName"}%
• OO Perl5 module, minimal changes to existing TWiki code

Note: This is intended as a tool for technical documentation teams and is therefore enabled on a per-web basis in our installation. It should only ever be a plug-in for that reason. It's too late to influence the development of TOC described in this page, but some of the ideas/code (such as the multi-topic section numbering and anchors and references) may find a use. Any takers?

-- CrawfordCurrie - 24 May 2001

Thanks Crawford for sharing this with the TWiki community! Over time we will take some code / ideas into TWiki, it is exactly what is needed when using TWiki for Book Authoring.

Note: Crawford's code is at TWikiForBookAuthoring, the docs at MotorolaExtensionsToTWiki.

Note: Moved Edgar's bug report of TOC not handling headings with identical names to TocFailsForIdenticalHeadingNames.

-- PeterThoeny - 16 Jun 2001

There is a HTML to pdf converter at http://www.easysw.com/htmldoc/ that I intend to try with the table of content concept.

-- BrianJohnson - 30 Jul 2001

BTW: supplied a hack in PrintUsingPDF based on htmldoc; it's very nice in that it creates an explorer type navigation tree on the left hand. One problem I see is keeping the header levels in sync, if you mix and match includes; mentioned that in another topic which I don't remember any more

Umpf - my browser goofed up this page, and I can't undo it. Sorry

(With previous releases of TWiki my NetPositive browser wouldn't even preview, so is this progress or not? Around sushi and maguro I see odd characters - maybe that's the problem?)

-- PeterKlausner - 30 Aug 2001

I rolled back the last revision since the browser Peter Klausner was using did some funny things to <pre> tags. Restored content.

-- PeterThoeny - 31 Aug 2001