create new tag
, view all tags
I don't know if some of you have seen the HyperPerl ( http://c2.com/cgi/wikibase?HyperPerl ) literate programming pages by WardCunnigham.

He has written a perl script to walk a set of topics to extract the StraightPerl code of his CommonWiki base version of wiki (see http://c2.com/cgi/wikibase ).

I find the idea fascinating, would it be better if we cooperatively develop TWiki itself in Twiki?

  • one routine per Topic
  • routine and variable names in WikiSyntax
  • walk the web from an ending point and collect all "preceeding" topics
  • collect all <PRE> code

-- AndreaSterbini - 13 Sep 2000

Agreed, literate programming is a fascinating idea. Start reading Ward's WikiWikiWeb program at http://c2.com/cgi/wikibase?WikiInHyperPerl

The nice thing about this idea is that program documentation and program code stays always in synch. JavaDoc uses this idea, and in Japan we used it for a CASE tool (nnHEADWAY/P, sorry, Japanese only) where I was the lead engineer. The CASE tool uses a charting system for documentation/design, and code fragments attached to chart elements for the program.

-- PeterThoeny - 14 Sep 2000

I like the idea as well. I've often thought that LiterateProgramming could be a way to tie different types of documentation together. LiterateWebProgramming, just takes it a step futher. It makes the document browsable by hyperlink as well so that function calls and variables become more tracable. You can also include use cases (or user stories), requirements, test cases, etc. in the same "document." I suppose you can keep multiple webs for a product. One for the production code, one for the test code, and one for the development code, etc.

I suppose that if you were doing ExtremeProgramming in a LiteralDevelopment environment, you could write the test case on the top of the page, and the code being tested on the bottom. Then the code generator could seperate the two parts and generate the code and the test cases.

-- JamalWills - 15 Sep 2000

Hmm, thinking again about literate programming I see one drawback for a team of developers. Lets suppose that people work together on the documentation and programming in just one web. This is fine for documentation, but for writing programs it is better if each developer has his/her own sandbox for testing out new features. Literate programming with one web means stepping on each others code: If there is a new bug I don't know if this is introduced by me or by someone else. Well, that could be solved with a web for each developer, but then there is a need for synchronizing the webs.

-- PeterThoeny - 17 Sep 2000

Keep a change area locally. Map the change area early into @INC (use lib). Then the walk of the Wiki gets the current release and sandbox changes are local. Update the Wiki once the code is accepted for release. With security controls this might work. It might, however, be as useful to generate wiki pages from POD in the source code.

-- GaroldJohnson - 09 Dec 2000

I've been doing Literate Programming for a few years now and in my experience the audience for the literate program itself is, (at least in my corporate and academic environments), different from the audience I see using Wiki pages. A LitProg developer gets a lot of mileage out of a very good editor (e.g. Vim or Emacs) which a web browser will never supply. Also, the versioning in groups is usually, as already noted, more sophisticated.

However, operator and user manuals, particularly by major software branch, seem to be ideal Wiki candidates. These could be enhanced with updated code documentation from a LitProg document generator (which might post attachments directly for related modules). I'm considering setting up programs that have their instructions and descriptions on a single Wiki page with the latest version (documentation and runnable form) and latest configuration files. That gives me all elements in one place with all the usual Wiki benefits, and allows me a place where humans and machines can collaboratively edit things like configurations.

At home, for my personal projects, I plan and do high level design on the Wiki, then link that to published versions constructed by my build program, and test routines which publish web-based test output. This allows the documentation to evolve in synch with the program; as design becomes code it usually make sense to move much of that content into code documents rather than leaving duplicates of it in overviews.

I think that the Wiki provides some unique and important documentation capabilities, but every project needs to select from a broad range of (fairly compatible) tools to construct a documentation strategy that is appropriate.

-- SimonClift - 16 May 2001

Any current interest / activity in putting the TWiki code on TWiki pages?

In my effort to learn Perl and TWiki, I started adding a lot of comments to (copies of) the TWiki programs. More than any normal developer would tolerate, I'm sure.

I had done something like this on my last VB project -- I added extensive comments to the code as long lines or paragraphs using Word. In Word those long lines wrapped and were highly visible, but I did detailed coding and debugging in the VB IDE editor where the comments did not wrap and were less obtrusive. I tried doing that with TWiki / Perl / Linux, but it's not working very well due to a combination of not having found the ideal tools and writing too many comments. (I'm using kwrite which provides syntax highlighting but does not have a "soft wrap" feature.) (Maybe the too many comments are Perl's fault, for doing so much in a regular expression substitution or whatever it's called wink

My current thought (writing this is partially a brainstorming effort) is to put the code on Twiki pages, with "standard" commments embedded, but create links from subroutine names and other TWiki identifiers and Perl keywords, to more extensive comments on other TWiki pages.

The comments I have added fall into two categories:

  • Newbie Perl comments -- comments that help a newbie Perl programmer understand the code (only, without understanding much about TWiki).
  • Newbie TWiki comments -- comments that help anybody (including a new or experienced Perl programmer, or a non-programmer) understand the details of what's going on in TWiki.

I'm learning a lot (I think) by doing this. Maybe it's something that every "new" programmer (new to a language or new to a particular program) has to spend some time doing, but I think that my comments (when polished at least a little) can make it easier for the next new Perl programmer or the next new TWiki developer. But, only if I can (1) make them easily accessible from the code (2) without driving the experienced TWiki developers crazy.

Although there would be some value in having these comments connected to a snapshot of the TWiki code at a particular point in time, it would be better if the comments were always linked to the current TWiki code.

I once made a superficial investigation of Doxygen and CTags to see if they could help me (about six months ago when AbiWord considered and then adopted Doxygen for their "standard" comments). A facility to create several different levels of comment ("standard", newbie to C++, newbie to AbiWord, maybe others) and a means to view the code with a selection of any or all of those comments would be nice. I couldn't find support for anything like that. (If you know of something, or something I've overlooked, please let me know.)

If TWiki does not become the repository for the TWiki code, I'd at least be looking for a simple and (semi-)automatic way to occasionally refresh a snapshot of the TWiki code on TWiki without having to recreate any links. (Identifiers that are currently links would automatically link to the same identifiers in the new code, identifiers that disappeared would lose their links, but maybe provide a report of which pages (of comments) are no longer linked. If I create links to lines of code (which I anticipate doing) and the line changes ... what do I do then? (I can foresee linking to specific lines of code for lines like the following, for which I wrote 10 long lines of comments:

if( /^\s+\*\sSet\s(ALLOWTOPIC|DENYTOPIC)$theAccessType\s*\=\s*(.*)/ ) {

It's hard (for me at least) to pick out any single element to tie all the comments to -- the keyword "if" would not be appropriate as the comments really constitute a piece by piece translation of the entire line.

I guess one way would be to insert a comment marker in the code, perhaps something like:

# pnc: See PncAccessPmCheckAccessPermissionC1

Where pnc stands for Perl Newbie Comment (and tnc could stand for TWiki Newbie Comment), and PncAccessPmCheckAccessPermissionC1 is the WikiWord link to the first comment (C1) in the checkAccessPermission subroutine of Access.pm.

I guess, now that I think about it, I could add these kind of references to the TWiki code even if TWiki is not the repository for the code -- a few of these would not be terribly obtrusive. It would still make more convenient reading if the code were stored in TWiki. With this approach (adding comment references in the code) maybe the approach of moving occasional snapshots of TWiki code to TWiki is workable -- maybe a snapshot after a major release?

With this approach, I would not put comments or comment markers in the code for keywords or identifiers -- I might include a general note that says something like:

There are WikiTopics containing comments on many (but not all) Perl keywords and TWiki identifiers in this program. Try capitalizing the first letter and going to a page with that name in this web. Many Perl "constructs" that look like punctuation ($/) also have dedicated pages -- try going to a page named like DollarSignSlash in this web.

Aside: The TWiki site that I am trying to implement, WikiLearn, is intended to help people learn things like Perl programming. What I'm describing might be overkill for TWiki -- it is very appropriate for what I envision for WikiLearn. And in what I consider to be a wiki-like approach, I would not feel bad if I made a partial effort at such documentation and never finished it -- someone else who finds it useful may still use it, and may carry it further if it advances their goals.

Thanks for reading this far. Suggestions appreciated (other than "go jump in a lake").

PS: This got longer than I originally intended. Maybe it should be moved to a different page. Suggestions?

-- RandyKramer - 03 Jul 2001

Just to add to the list of possibilities (however remote), LXR came to my attention. (I guess I had seen LXR and references to it, but I don't think I tried to use it more than once and I didn't think about it as a possibility.) I haven't retried it recently, but I will. And it's currently only for C/C++.

AFAICT, LXR is a tool to easily create hyperlinked cross-references in a code base and then navigate in the code using those cross-references. (It is written in Perl, and I've subscribed to the mail list -- maybe someone there knows of a similar tool for Perl.)

My thought is that (if LXR worked for Perl), I could import all the code into LXR and then create additional files that contain newbie Perl and newbie TWiki comments. Then, if you want those comments for an identifier or function, you jump to an appropriate cross-reference in one of those files.

The cross-references appear to deal with identifiers (including function names) -- I suspect it doesn't deal with keywords, nor cross-references to (entire) lines of code, but that might be handled by adding "comment references" as mentioned above.

They say (in my words) that the parsing is rather "loose", which can be good or bad. It's good in that perhaps currently non-working code can also be automatically cross-referenced, it's bad in that it has trouble dealing with identifier scope, so that identifiers with the same name but different scope are not distinguished. (I wonder if it would be easier to parse Perl properly for an "ancillary" function like this, or if it would be easy to grab the Perl parser from the Perl interpreter and utilize it? (just daydreaming))

I have to go back to where this caught my eye today -- the project that mentioned it said that their entire code base was now in LXR -- I wonder if that means that LXR is now their code repository? (Probably not -- I tend to jump to conclusions very quickly sometimes. Now, how do I find that reference again? I think it came while reading LWN.)

Ok, I found it, it had to do with KDE, and this is quoted from LWN:

KDE moved to LXR. The entire KDE source tree has been moved to LXR. According to Kurt Granroth, this is a major improvement in cross referencing. "If I go to the KConfig class in LXR, though, I see that all objects and methods in the file are hyperlinks. If I click on KConfig, I get a list of where it is defined, where it is declared as a forward declaration, and where it is referenced (in 939 files!). I can click on any of those links to go directly to where it is used."

-- or two of those references could be to "newbie-ized" comments for new TWiki developers or Perl programmers. (Some sort of trick may be required, I'm not sure that cross-referencing to comments is a standard feature.) (It doesn't say that LXR is the code repository for KDE -- I'll try digging deeper.)

-- RandyKramer - 04 Jul 2001

Anybody using or familiar with a "code folding" editor (other than Emacs)? I'm thinking of one that works for Perl code, and I'd like to learn what must be put in the code to support the folding operation.

(And since I'm looking for an editor (for Linux), I'd like to find one with, among other things, syntax highlighting, soft word wrap, and code folding.)

I took a glance at a little bit of information about Emacs, it seems they do something with inserting extra ";"s, "{"s, and "}"s to support code folding (I'm still reading). I wonder if we could put something in TWiki code to do something similar. The problem is, anybody that doesn't use a code folding editor would have to wade through all the comments that were there. That would not be nice.

Is code folding (in an editor) an idea whose time has come and gone?

Aside: I wonder which came first -- Word's collapsible outlining mode (or some other word, thought, or outline processor's), or a programming editor's code folding?

code2html might be another useful tool.

-- RandyKramer - 07 Jul 2001

You might be interested in the ViM TWiki syntax file. I've written a patch that adds folding support; I've contacted the author of twiki.vim about getting it included in the source. Feel free to contact me with any further details.

-- IanTegebo - 19 Feb 2006

I find this stuff fascinating. I agree, it would be cool to have a plug-in to allow you to use TWiki to comment on the code. In many ways I see TWiki as a mechanism to componentise and structure knowledge, just as we use programming to componentise, structure and automate processes to be followed. With TWiki the mind is left to wander and re-link as it sees fit.

Keep it coming!

-- MartinCleaver - 10 Jul 2001

Sounds like you're approaching the single idea of "multifaceted software" from different directions. You can find an article in about it in Software Development Magazine at

"Through the Looking Glass (July 2001) Subtle signs point to a marked transformation, a disruptive technology on the horizon." by Grady Booch http://www.sdmagazine.com/articles/2001/0107/0107i/0107i.htm

This is clearly reflected in Microsoft's research called Intentional Programming. http://www.aisto.com/roeder/ip/ holds some info and specifically http://www.research.microsoft.com/ip/May99/Disrupt.htm IBM, Xerox, MIT & others are working on the idea too.

In a nutshell explanation (my version), Folded Code, Class Browsers, Literate Development, Grep Listing, Database Triggers, Code Files in Directory Hierarchies, and UML, 3GL, 4GL languages, Functional Programming are all various views of the logic and code. Editing is still done at the text level (or in UML by some drag & drop).

In "multifaceted software", instead of only allowing a limited set of views provided by a specific product's vendor, let the developers define any view they want. Edit the logic in any view at the level of describing the change. This is analogous to SQL's representation and change of data. You don't copy-paste SQL views of data. You issue commands to describe the change at the level of the abstractions of the data. Would you trust financial application users to copy-paste cells of data to create all their reports? Why do programmers?

Views of the abstract representation of code can also have scope beyond what object oriented programming can. You can manipulate (edit) an intention, concept, or representation at the micro level, say, part of a statement or even an attribute of a variable name. You can edit an intention, concept, or representation at a macro level, say, of where the code is eventually to be stored (local, host, mobile agent), how fast the code is, by how the object is called, or across many selected objects irrespective of where they appear in the hierarchy. That's the way programmers think.

-- SocinianClarke - 10 Aug 2001

1: Socinian,

Thanks for these references. (I meant to thank you earlier -- I found them around the time you made them and started to explore them -- I need to do a reread.)

2: I found a reference to embedding code in HTML (http://www.digitalmars.com/d/html.html) which fits into this discussion (and which has other roots as well, I'm sure). I have more notes about it on my private (someday to be public) twiki, WikiLearn, at topic Programming.EmbeddingCodeInHTML.

It mentions or leads me to the ideas of:

  • maintaining the code (with comments) in HTML (or XML, XHTML, or something similar) (with code enclosed in <code> &lt,/code> tags, or maybe not enclosed in any tags)
  • finding (or inventing) an HTML (XML, XHTML, etc.) tag which includes a level attribute (either in numbers or codes like PNC (for Perl Newbie Comment))
  • commenting the TWiki programs to my heart's content, marking each comment with the tag with the appropriate attribute (PNC, TNC, etc.)
  • a preprocessor which strips out comments and HTML markup before passing to the compiler (or interpreter)


  • near term: create a preprocessor that converts the tagged comments to and from HTML comments by enclosing them in <!-- --> tags, which has the effect of making them visible or invisible in a (web) browser.
  • long term: create a (web) browser that can recognize those tags, and optionally display or hide text enclosed in those tags
  • Aside: both the preprocessor and the browser should allow hiding or display of arbitrary combinations of the tagged comments, for example, TNC and PNC, TNC only, PNC only (and more combinations if more categories exist)
  • ideally, editing (of code or comments) would take place in the same browser so that only one source document is maintained -- editing should be allowed while some or all of the comments are hidden, but there may then need to be some automatic means of determining where new lines of code are inserted relative to all existing (but possibly hidden) comments (or simply display all comments as part of the process of inserting a new line).

Associated with this, elementary Perl Newbie Comments (like explanations for each Perl keyword) would be in separate files, called up by a mechanism something like help or hyperlinks).

Clearly this is not a totally thought out approach, but I do want to remind myself here to consider Peter Thoeny's concern about programmer's needing their own sandbox for development (17 Sep 2000). Without much thinking, it would seem that CVS should provide a solution, but I've done no thinking at all about how. With luck, the CVS features of TWiki, in conjunction with each developer maintaining his own private TWiki with the TWiki code, and some means for checkin and checkout to the "master" TWiki might be workable.

-- RandyKramer - 17 Aug 2001

Software Development Magazine has an article on Aspect Oriented Programming from Xerox PARC called AspectJ : http://www.sdmagazine.com/documents/s=1123/sdm0109e/0109e.htm

Here are the Xerox PARC sites: http://www.aspectj.org and http://www.parc.xerox.com/csl/projects/aop/

-- SocinianClarke - 27 Aug 2001

Brian Palmer on comp.programming.literate made me aware of Leo, http://personalpages.tds.net/~edream/FAQ.html#anchor128891. It looks very interesting. It's a browser / editor for literate programming based on outlining. Files are stored as XML.

Anybody seen / used it? I tentatively plan to try to put some of TWiki in it after the next official release (Spring 2001). If anybody who has used it has any suggestions on how to break TWiki into chunks (or whatever they call it), let me know. Tentatively I'm thinking of starting with the view program, and making each subroutine a separate chunk / heading in the outline. Ideally (I think) I should start with the "top level" TWiki program, but, AFAICT, there really isn't one, unless I "focus" on, for example, viewing a page. Then I can treat view as the top level program.

BTW, it seems that this blends some aspects of the intentional programming that Socinian mentions above (at least, AFAICT by "browsing" some of Charles Simonyi's papers.

-- RandyKramer - 30 Aug 2001

I've spent a little time learning about Leo (including Leo1 and Leo2). For various reasons (which I'll discuss on a different page, later, -- actually, first on my home TWiki), I think Leo1 is better suited for this task than Leo2. (I'll also note that neither Leo1 nor Leo2 takes quite the approach I was thinking of in documenting a program in XML with the ability to hide different levels of comments. It would be helpful if programming editors existed which could work in XML, and hide/ignore comments labeled with a particular tag/attribute.

-- RandyKramer - 13 Sep 2001

See TWikiInLeo (very preliminary) and Wikilearn.TWikiVariablesForTWikiNewbies (also preliminary).

-- RandyKramer - 30 Sep 2001

Interesting page, well hidden. We need maybe a link to LongForgotten pages? smile

RandyKramer suggested to use code2html. I used and liked PerlTidy. In two steps, it will create aligned source code (with my favorite feature, } #end if ending comment at the end of if syatement, etc.), and then can create syntax-colorized HTML file to be published on web. perltidy has highly customizable setting for both code formatting and syntax-colorizing.

For editing perl code, I use MultiEdit. It can collapse source code based on reg. expressions, on outline (which is perltidy-ed), has tags ("show source for this sub'), can "open file under cursor" and "grep file for word under cursor". MultiEdit has C-like internal language. In my previous job, colleague implemented a MultiEdit macro, inserting change history and commend with signature and date to each changed line, and another one, synchronizing two folders (and displaying recent change history).

ME is not free, but well worth $100.

-- PeterMasiar - 02 Apr 2002

Comments cut to PerlTidy

-- MartinCleaver - 02 Oct 2004

Someone may have mentioned it above — but the web edit interface to TWiki wouldn't be enough for editing. Also you need branching (as someone says). Now TWiki uses RCS at the moment, and RCS does branching. (You could replace RCS with CVS or Subversion, but that's another issue). You would need some way to edit the topics using Emacs or an IDE. Perhaps WebDave could be used.

-- MartinEllison - 02 May 2002

The Scite editor, based on Scintilla, is a good code folding editor that happens to know Perl (and dozens of other languages) out of the box. Multi-platform. Includes source. Everyone's favorite price: $0.00 http://www.scintilla.org

-- DavidLeBlanc - 04 May 2002

I've got an idea for a plug-in that would allow static, wiki-formatted content to be browsed within the TWiki framework. This would allow LiterateProgramming to be done using conventional tools, but the output documentation to appear in the Wiki. The idea is in StaticWikiFormatContentPlugin if anyone wants to help brainstorm on this.

-- SimonClift - 30 Apr 2003

Edit | Attach | Watch | Print version | History: r32 < r31 < r30 < r29 < r28 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r32 - 2006-02-19 - IanTegebo
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.