Developing Extensions with BuildContrib and TWikiShellContrib

Preparing to work

BuildContrib makes some basic assumptions about the enviroment:

• The TWikiExtensions is not being developed your plugin in a live TWiki installation (which is usually a really bad idea), but are instead are developed in a separate directory tree, usually - but not always - a CVS checkout area.
• The TWikiExtensions has a directory for itself outside the twiki root
  • (ok, this is not an assumption, but if you try to generate a manifest for the TWikiExtensions from a twiki installation it'll end up with a lot of useless entries. Also, it's a good habit to acquire ;))
  • Please explain this - MC
• The enviroment variable TWIKI_LIBS has a colon-separated path list of Perl libraries needed to satisfy any of the TWikiExtension external dependencies. These may point to any directory.
• The enviroment variable TWIKI_HOME point to the root of your test TWiki installation.
• Your module follows the standards for plugins and contribs i.e. it
  1. Has a topic in the data/TWiki directory
  2. Has a perl module in the lib/TWiki/Plugins or lib/TWiki/Contrib directory
  3. Has a sub-directory alongside the perl module that contains the extra .pm files specific to the module

Normal Development Cycle

Let's illustrate how BuildContrib fits in the development lifecycle, step by step.

Create the Directory Structure

TWikiExtension

The recommended directory layout & content for TWikiExtension development is:

|- twikiroot
|  |- twikiplugins
|  |  |- TWikiExtensionName
|  |  |  |- bin
|  |  |  |- lib
|  |  |  |  |- TWiki
|  |  |  |  |  |- (Contrib|Plugins)
|  |  |  |  |  |  |- TWikiExtensionName
|  |  |  |  |  |  |  |- build.pl
|  |  |  |  |  |  |  |- MANIFEST
|  |  |  |  |  |  |  |- DEPENDENCIES 
|  |  |  |  |  |  |- TWikiExtensionName.pm
|  |  |  |- data
|  |  |  |  |- TWiki
|  |  |  |  |   |- TWikiExtensionName.txt
|  |  |  |- pub
|  |  |  |- tests
|  |  |  |  |- unit
|  |  |  |  |  |- TWikiExtensionNameSuite.pm
|  |  |  |- templates

Note: twikiplugins is the checkout area from the plugins repository (either from CVS or SVN)

• Raf - we're going to deprecate CVS - can you rework these out?

To build the initial MANIFEST file, invoke:

perl build.pl manifest

A "tentative" MANIFEST file will be sent to STDOUT, so it can be copy/pasted or redirected to the MANIFEST file and edited as needed. (This uses BuildContrib)

The MANIFEST file should minimally contain:

lib/TWiki/(Contrib|Plugins)/TWikiExtensionName.pm
data/TWiki/TWikiExtensionName.txt
tests/unit/TWikiExtensionNameSuite.pm

Note that none of the special files used by BuildContrib is in the MANIFEST, because they are usually not distributed to end-users.

For a complete description of the format of the MANIFEST and DEPENDENCIES files, check Here.

"Installing" the TWikiExtension

To test the TWikiExtension, it must be "installed" into the twikiroot. There are two options to do it:

1. Make the change in the plugins checkout area, and deploy them to the test TWiki installation OR
2. Deploy the plugin to the test TWiki installation, make the changes "live", and then copy back the modifications to the checkout area

BuildContrib can help in the first case. Invoking

perl build.pl install

will "install" the plugin (along any unsatisfied dependency) in the twiki installation pointed by the enviroment variable TWIKI_HOME.

Adding or Removing files

Often, files are added or removed from the distribution. For example, some plugins have utility methods not in the main .pm file but in separate module. Sometimes the plugin topic is not enough so additional topics are shipped with the plugin.

In those cases, the MANIFEST file must be updated with the adds/deletions or BuildContrib will not work properly.

Running tests

If Test::Unit is installed in the system, it's possible to run the unit test with BuildContrib. Just Invoke:

perl build.pl test

And it will try to run automatically the tests/unit/SamplePluginSuite.pm module.

Preparing the release package

To package the extension for a release, invoke

perl build.pl release

and it will create a .zip and a .tar.gz files based in the content of the MANIFEST file and an installer called TWikiExtensionName_installer (ie. SamplePlugin_installer). All these files will be created in the "root" of the plugin directory. (ie: under twikiplugins/SamplePlugin). As part of the release process, the TWikiExtension topic is processed an the following tags will be replaced:

• %$MANIFEST% - TWiki table of files in MANIFEST
• %$DEPENDENCIES% - list of dependencies from DEPENDENCIES
• %$VERSION% version from $VERSION in main .pm
• %$DATE% - local date
• %$POD% - expands to the POD documentation for the package, excluding test modules.

Uploading the changes to TWiki.org

perl build.pl upload

and BuildContrib will create the release package, and upload it automatically to the proper place in TWiki.org.

TWikiShellContrib Quick guide to develop TWikiExtensions

The default CommandSets bundled with TWikiShellContrib can be used to simplify even more the development process, by complementing the operations already performed by BuildContrib.

Preparing to work

TWikiShellContrib ask the user about the enviroment where it's running (where the twiki root is located, how the directories are configure, etc), but it makes two assuptions:

• The checkout area is called "twikiplugins" and sits under the twiki root
• MANIFEST and DEPENDENCIES are in the plugin "root"

Note that no enviroment variables need to be set for TWikiShellContrib to work.

Normal Development Cycle

To illustrate how TWikiShellContrib fits in the development lifecycle, let's follow the same process as with BuildContrib.

Create the Directory Structure

TWikiExtension

The directory structure is as follows:

A typical directory layout & content for TWikiExtension development is:

|- twikiroot
|  |- twikiplugins
|  |  |- TWikiExtensionName
|  |  |  |- MANIFEST
|  |  |  |- DEPENDENCIES 
|  |  |  |- bin
|  |  |  |- lib
|  |  |  |  |- TWiki
|  |  |  |  |  |- (Contrib|Plugins)
|  |  |  |  |  |  |- TWikiExtensionName
|  |  |  |  |  |  |  |- build.pl
|  |  |  |  |  |  |- TWikiExtensionName.pm
|  |  |  |- data
|  |  |  |  |- TWiki
|  |  |  |  |   |- TWikiExtensionName.txt
|  |  |  |- pub
|  |  |  |- tests
|  |  |  |  |- unit
|  |  |  |  |  |- TWikiExtensionNameSuite.pm
|  |  |  |- templates

Note: twikiplugins is the checkout area from the plugins repository (either from CVS or SVN)

To create this structure, invoke:

perl twikishell plugin create TWikiExtensionName

and TWikiShellContrib will create the directory structure with empty skeletons for all the files. The MANIFEST file is created automatically.

• Raf - are these files placed under twikiplugins or in the users' live site?

The MANIFEST file should read:

lib/TWiki/Plugins/TWikiExtensionName.pm
data/TWiki/TWikiExtensionName.txt
tests/unit/TWikiExtensionNameSuite.pm

Note that none of the special files used by BuildContrib is in the MANIFEST, because they are usually not distributed to end-users.

Live editing of a TWikiExtension

The invocation:

perl twikishell plugin develop TWikiExtension

Deploys the plugin to the working TWiki installation, in which the programmer can make the changes "live". Another command helps pushes the modifications back to the checkout area.

plugin develop copies all content named in the MANIFEST of the plugin directory into the twiki installation. Also, two files called TWikiExtensionName.MF and TWikiExtensionName.DEP are created in the twiki root. TWikiExtension.MF will have a list of all the copied files, and should be identical to the MANIFEST file. Similary, the TWikiExtension.DEP should be identical to the DEPENDENCIES file.

Now, all the changes can be made "live".

After all the changes, invoke:

perl twikishell plugin putback TWikiExtension

and it will copy all the files listed in TWikiExtension.MF back to the checkout area, updating the MANIFEST and DEPENDENCIES with the content of TWikiExtension.MF and TWikiExtension.DEP respectively.

Adding or Removing files

When adding or removing files from the package, the TWikiExtension.MF must be updated so the proper MANIFEST file can be generated later.

Running tests

If Test::Unit is installed in the system, invoking

perl twikishell runtest TWikiExtensionSuite

will try to run automatically the tests/unit/TWikiExtensionSuite.pm module.

Preparing the release package

To release the package, TWikiShellContrib interfaces with BuildContrib, which handled the release process.

To create the release file, invoke

perl twikishell build TWikiExtension release

This will call automatically the proper build.pl script with the release target

Uploading the changes to TWiki.org

In the same way as releasing the package, invoking

perl twikishell build TWikiExtension upload

will call automatically the proper build.pl script with the upload target.

Additional Operation: Packaging the development version

If for some reason you need to package the whole development version of a TWikiExtension (including the MANIFEST, DEPENDENCIES,etc), invoke:

perl twikishell package TWikiExtension

This will create a tar.gz file in your current directory containing all the files for the plugin.

Building shortcuts

perl twikishell shorcuts

To reduce the amount of typing needed, use twikishell shortcuts to create a set of shellscripts for your path. Then you can do:

• plugin create TWikiExtension
• plugin develop TWikiExtension
• runtest TWikiExtension or build TWikiExtension test
• plugin putback TWikiExtension
• (optionally) build release TWikiExtension
• (optionally) package TWikiExtension
• build upload TWikiExtension