Feedback on PublishWebPlugin

First version posted, enjoy.

I wanted to be able to automate the website maintenance process as much a possible. With this light-weight Plugin, a team can maintain a website on an internal TWiki; a web page get generated/updated each time someone saves a topic in a dedicated Publish web. There is no indication that the HTML pages are generated, all references to the internal TWiki are removed.

Possible enhancements:

• Add a default publish skin for customization
• Add a feature to regenerate all HTML pages, e.g. after a skin modification
  • DONE 15 Feb 2006
• Introduce subdirectories for the attachments to avoid namespace issues (but that would reveil the TWiki web structure)
• Add a feature to update selected topics via scheduled task, allowing automated publishing processes
  • DONE 15 Feb 2006, possible with:
    wget http://example.com/view/Publish/WebPublish?action=publish;ptopic=Topic_Name

The PublishContrib is related, it can also generate static HTML pages.

-- PeterThoeny - 13 Feb 2006

New Plugin version 15 Feb 2006 posted:

• Added %PUBLISHWEB{"publish" topic="Name"}% feature to re-publish topics on demand
• Fixed issue where WEBPUBLISH setting had no effect. Caused by identical names of WEBPUBLISH setting and %PUBLISHWEB{}% variable. (I fell into this trap already with the BlackListPlugin)

-- PeterThoeny - 15 Feb 2006

The Plugin makes it very easy to maintain websites, anyone can easily create/update content, especially with the WYSIWYG editor. I think this Plugins opens up new deployment opportunities for TWiki:

• Maintenance of SMB websites (small and medium-sized businesses)
• Maintenance of family and community websites

The missing piece is a configurable publish skin that is themeable. I envision a skin where you can choose from different themes (colors, borders, styles, banner images etc). Anyone interested in helping out?

-- PeterThoeny - 15 Feb 2006

I'm having a hard time understanding what this plugin does than PublishAddOn doesn't do. Can you clarify?

-- CrawfordCurrie - 16 Feb 2006

This Plugin offers a clean and simple way to publish web pages. The main iching factor was usability. Unlike the PublishAddOn, publishing is a behind-the-scene process (no explicit publish step), does not depend on an intermediate ZIP files to unpack and copy, has no other Perl dependencies, and is much less CPU-intensive on the server.

-- PeterThoeny - 16 Feb 2006

OK, but I wish you had raised this with me. The zip step was added to the PublishAddOn by users who demanded it. The zip step is the CPU-intensive piece; as you have discovered, if you don;t zip, it's nowhere near as costly. The original GenHTMLAddon used to do pretty much what you have described for PublishWebPlugin, so I get a strong feeling of deja vu and duplicated effort,

-- CrawfordCurrie - 16 Feb 2006

Hmm, I do not see this as duplicated effort since PublishAddOn and PublishWebPlugin serve different needs. I did not look at the GenHTMLAddon since it is no longer maintained (and it does not do the automatic behind-the-scene publishing).

-- PeterThoeny - 16 Feb 2006

I'm getting an error when tryign to edit TWiki.PublishWebPlugin file to change the skin:

During save of TWiki.PublishWebPlugin an error was found by the version control system. Please notify your TWiki administrator.

=RCS: /usr/bin/rcs -q -l %FILENAME|F% failed: = 

Is this plugin related to you think or twiki related? I'm trying this on a fresh install of twiki 4.0.1. Thanks!

-- JeffLink - 27 Feb 2006

Most probably a file permission issue. Make sure that the webserver user can write to the .txt and .txt,v files. Also check the rcs lock user, should be same webserver user or unlocked for TWiki 4.0.

-- PeterThoeny - 27 Feb 2006

And in reply to myself... with Peter's guidance... needed to change "webuser" to the user that owned the file in the PublishWebPlugin.txt,v file.

-- JeffLink - 27 Feb 2006

TOC does not seem to be converting to file.html anchor. TOC links are linking to the bin/view/etc... reference.

Also... I have twiki installed on the public_html folder. so domain.com/pub is the pub dir. I have the hardcode path to the published html files to domain.com/kb/ ($publishPath = "../kb";). When using using the breadcrumb, the links are domain.com/file.html not domain.com/kb/file.html. Do I have some setting wrong? Thanks in advance!

-- JeffLink - 28 Feb 2006

Peter, If you put some kind of authorisation option (workflow) into the publish process, you're very close to a "real" CMS.

-- MarcusLeonard - 01 Mar 2006

Thanks Peter: it help us make a "PermaLink" on old content without complex rewriting configuration.

but I meet the same subdir publishing problem like Jeff Link: if you pubish docs into a non root directory(like "/foo/"): the page.html links ot other topic(Bar) is still to root "/bar.html"

Should we define a $destPublishDir?

Also... I have twiki installed on the public_html folder. so domain.com/pub is the pub dir. I have the hardcode path to the published html files to domain.com/kb/ ($publishPath = "../kb";). When using using the breadcrumb, the links are domain.com/file.html not domain.com/kb/file.html. Do I have some setting wrong? Thanks in advance!

-- CheDong - 05 Mar 2006

OK, there is clearly a need for a destination URL other than the root.

-- PeterThoeny - 05 Mar 2006

Potential bug in attachment link described in Support.PublishWebPluginAttachmentLinks.

• PublishWebPluginAttachmentLinks has a fix for attachment link issue (by WolfgangSlany) -- PeterThoeny - 08 Sep 2006

-- PeterThoeny - 01 Apr 2006

Idea: 2, Design: 3, Implementation: 2, Documentation: 3, Examples: 3, Cleanliness: 2

Duplicates PublishContrib. Requires the administrator to alter the .pm file.

-- MartinCleaver - 18 Apr 2006

The PublishContrib did not exist at the time I wrote the PublishWebPlugin. The PublishAddOn (PublishContrib's predecessor) had a different purpose: Publishing content as HTML (optionally ZIPed) for offline reading. It did not serve my needs from a usability point of view. Unlike the PublishAddOn, publishing in the PublishWebPlugin is a behind-the-scene process (no explicit publish step), does not depend on an intermediate ZIP files to unpack and copy, has no other Perl dependencies, and is much less CPU-intensive on the server.

Also, the .pm file only needs to be altered for non-standard installations.

-- PeterThoeny - 18 Apr 2006

Just to clarify: first there was GenHTMLAddon dating back to 1 Oct 2001, then PublishAddOn 7 Jan 2003 which has been renamed to PublishContrib 27 Feb 2006. PublishWebPlugin released 13 Feb 2006.

There is no more need to generate a zip archive first. PublishContrib supports tar and pdf too now.

The PublishWebPlugin can generate static html on each save but also allows dedicated publishing of a web. Even though I'd prefer to publish a web in one step to get a consistent transition from one publication to the next, adding publish-by-save would have found a reasonable home in the PublishContrib also.

Remaining features are largely overlapping.

Peter, you mentioned usability issues with the PublishContrib. Can you be more precise?

-- MichaelDaum - 19 Apr 2006

I moved MartinCleaver's and MichaelDaum's appraisals from the appraisal topic to here because both did not actually install the Plugin, and because there is some discussion going on on their feedback.

First a clarification: I am not here to compete, I strive to share ideas and support the community in any way I can.

On the usability issue, I mentioned this before: I had a need for a behind-the-scene publishing process. Set up once, then forget about the publishing process. GenHTMLAddon and PublishAddOn did not have that, and the PublishAddOn was much too heavy on the server. Hence the reason to create this Plugin that does something very specific, and does it well (Unix philosophy). It also is lightweight and has no dependencies. You can see the result at http://www.structuredwikis.com/.

-- PeterThoeny - 19 Apr 2006

Peter, I did install your plugin. I even opened up your code. I just say that these two separate developments should converge: PublishContrib should provide a clean api to publish webs and topics and a PublishPlugin will use that, i.e. in its afterSaveHandler. Admitted, there needs to be some cleanup in the PublishContrib (like not hardcode print feedback). But that's pretty much all afaics. You can implement any usability policy you like in the plugin part but use publish services in a shared library.

And -- pardon me -- I will reestablish my votes on PublishWebPluginAppraisal.

The results of the current PublishContrib output can be seen at http://www.wikiring.com.

-- MichaelDaum - 20 Apr 2006

I did install the plugin and even opened up the code, but uninstalled it again in favor of the PublishContrib. I decided to fixed PublishContrib to work with the NatSkin. I did not try PublishWebPlugin using the NatSkin. I expect it to fail for the same reasons PublishContrib failed before. This is part of a more fundamental problem rebuilding the preferences cache for each topic. I don't have the time to fix that for the PublishWebPlugin also. I hope you understand, Peter.

• The PublishWebPlugin does not need to build a preferences cache since it acts on the save callback, e.g. publishing is restricted by whatever access permission is set. -- PeterThoeny - 20 Apr 2006

-- MichaelDaum - 20 Apr 2006

In Peter's defence, he had a pretty clear idea of what the PublishAddOn could do before he embarked on this development, as he had funded improvements to the add-on. It's rather a shame that (I guess) he missed my statements to the effect that the poor performance was due to the compression, which was trivial to make optional, and that the contrib had been renamed and checked into subversion some months ago. I didn't re-release at that time because I had new features queued for TWiki-4.

PublishWebPlugin and PublishContrib don't work quite the same way. PublishWebPlugin has features similar to %SECTIONs to support conditional publishing of topics, which PublishContrib doesn't have (PublishContrib relies instead on the existing TWiki mechanisms, such as %STARTINCLUDE).

Ehm, not quite correct: PublishWebPlugin uses STARTPUBLISH....STOPPUBLISH whereas PublishContrib uses <nopublish> ... </nopublish>. Kind of same idea done differently. Both makes sense.

-- MichaelDaum - 20 Apr 2006

PublishContrib was formerly limited to zipfile output, though thanks to recent support for improvements from clients, it now supports plain html publishing, as well as tgz, zip, pdf and postscript output formats.

FYI, note that the PublishContrib has always been able to run in a batch mode (it uses a "publishing topic" to control the batch publishing process). The documentation on this use mode is poor, however.

Peter could have kept PublishWebPlugin internal for his own use; instead he chose to publish it for the benefit of the community, despite the obvious overlaps. Rather than criticising, we should all welcome any work that seeks to improve on existing technologies!

-- CrawfordCurrie - 20 Apr 2006

I do welcome the plugin. I even stopped to wonder the contrib was not called a plugin.

It always saddens me when we do exploration when the concepts are stable enough that we should be doing consolidation. Without ExplorationThenConsolidation there is not requisite variety to keep innovating or not sufficient sanity to keep the community efforts from disapating chasing all the offshoots.

Had the plugin been issued by someone on the periphery of development I'd have been less disappointed. I'd still now be pushing for consolidation though.

Let's focus on the go-forward actions.

-- MartinCleaver - 20 Apr 2006

Thanks Crawford for putting things into the right perspective. Again, I am not here to compete, I am here for sharing ideas, code, docs with the TWikiCommunity.

There is no one size fits all. I am glad that Micha found a solution that fits his needs. Although I must admit his shift in opinion took me by surprise: At the beginning a very positive e-mail on this Plugin, and now a very negative appraisal.

I removed Martin's appraisal again since he did not actually install this Plugin and since his opinion is stated on this dev topic.

-- PeterThoeny - 20 Apr 2006

Peter, certainly I am all positive about every contribution. But after evaluating both publish solutions, looking up the code and testing it with the NatSkin I had to revise my initial ratings. That's normal.

-- MichaelDaum - 21 Apr 2006

For whatever the feedback is worth: I just considered PublishContrib and PublishWebPlugin and wound up going with PublishContrib for now for the following reasons:

• The attachment namespace collision problem with PublishWebPlugin
• PublishWebPlugin converts topic names to lowercase filenames (why is this? it makes it difficult to have the published site mirror the real site exactly)
  • Reasons: URL path are usually lowercase (also easier to spell over the phone). It is irrelevant for the consumer that the pages are generated from the case sensitive TWiki. -- PeterThoeny
    • This assumes that the "consumer" and the users of the unpublished Wiki are two different sets of people. They are not in my case, so this is confusing to them. And Web URLs are typically case-sensitive. Perhaps the solution here would be to make TWiki case insensitive all together (although this would mean getting rid of WikiWords). -- MarkMontague
• WikiWord links are ignored (I really dislike the idea of WikiWords, as they cause a lot of confusion for novice users, and I'd love to see WikiWords eliminated from TWiki entirely in favor of double bracket links only, but right now I feel that PublishWebPlugin not supporting WikiWord links would break too much existing content and add even more confusion around the whole concept of WikiWord linking)
  • Note: You can disable autolinking for a whole TWiki site or web. See TWikiPreferences. -- PeterThoeny
    • Yes, but some web administrators want their webs published and yet have chosen to use autolinking. PublishWebPlugin breaks this for them in the published version of their web. Basically, my point both with autolinking and case-sensitivity is that most of the reasons for me not to use PublishWebPlugin at this point is that, unlike PublishContrib, it makes different assumptions from those that TWiki itself makes. -- MarkMontague

On the other hand, the publish-on-edit functionality is wonderful, and it would be great to switch to PublishWebPlugin at some point in the future.

I hope this feedback is helpful. Thanks for everything, Peter, and also thanks to everyone who has worked on PublishContrib.

-- MarkMontague - 24 Apr 2006

Re: OK, there is clearly a need for a destination URL other than the root. -- PeterThoeny - 05 Mar 2006

• We would need this urgently as we are running several twikis in parallel on one webserver (some for courses, some for projects, and some for subgroup intranets; they all have the same base url). How complicated would it be?

-- WolfgangSlany - 26 Apr 2006

A configurable destination directory is trivial to implement.

Is there a need to publish from multiple webs to multiple destination directories?

-- PeterThoeny - 27 Apr 2006

Re: Is there a need to publish from multiple webs to multiple destination directories? -- PeterThoeny - 27 Apr 2006

• Not for me.

-- WolfgangSlany - 28 Apr 2006

BTW, is there some way to get notified when this page changes? The NotificationPlugin should be perfect for it, but it seems not to be activated here.

-- WolfgangSlany - 28 Apr 2006

Outgoing e-mail is not working on twiki.org. This will be soon fixed, together with TWiki 4 upgrade.

-- PeterThoeny - 28 Apr 2006

Found a bug? If topics have numbers in them, those numbers are lost. e.g. Something2006 is turned into something.html - the '2006' is dropped. I found this also applies to single numerals: SomethingElse9Fails -> somethingelsefails.html, and is true whether the numbers are in the middle of the string or at the end.

This fixes it:

--- lib/TWiki/Plugins/PublishWebPlugin.pm       Wed Feb 15 15:55:40 2006
+++ PublishWebPlugin.pm Sat May  6 19:02:29 2006
@@ -258,7 +258,7 @@
     # 'file':   '/file/path/to/topic_name.html'
     # 'label':  'Topic Name'
     my $text = lc( $topic ) . '.html';
-    $text =~ s/[^a-z_\.]+//go;
+    $text =~ s/[^a-z0-9_\.]+//go;
     $text =~ /(.*)/;
     $text = $1; # untaint
     if( $type eq 'url' ) {

This is from the current version of the plugin (DL'd today)

-- JasonPeacock - 06 May 2006

Also, when first installed and used, the plugin generates RCS errors b/c the ,v files have a different user associated with them. Maybe it would be better to not include the ,v files in the distribution? Or is there a different way to fix that problem - I notice that all other ,v files in the TWiki distro don't have that problem. (I'm not an RCS expert...)

-- JasonPeacock - 06 May 2006

Thanks for the bug report, I'll incude the fix in the next release.

On ,v files, topics are in Cairo format (mostly compatible with Dakar). Cairo's ,v files have locks, Dakar not. You get the issue described in Dakar. A proper fix is to make Dakar more forgiving to locks, e.g. fully support Cairo files.

-- PeterThoeny - 07 May 2006

See KeepRcsFilesUnlocked for a discussion of this. Apparently there are both issues with conflicting users and performance costs. I'm sure CDot can discuss this more thoroughly, however.

-- MeredithLesly - 07 May 2006

here there is a patch to get publish url out of root website

in TWiki/PublishWebPlugin

• set PUBLISHURL = /new/url/base

or

• set PUBLISHURL = http://www.mysite.com/new/url/base

--- lib/TWiki/Plugins/PublishWebPlugin.pm   2006-02-15 17:55:40.000000000 -0300
+++ ../lib/TWiki/Plugins/PublishWebPlugin.pm    2006-05-10 02:26:10.000000000 -0300
@@ -72,6 +72,7 @@
     $excludeTopic =~ s/,\s*/\|/go;
     $excludeTopic = '(' . $excludeTopic . ')';
     $homeLabel    = TWiki::Func::getPluginPreferencesValue( "HOMELABEL" ) || "Home";
+    $publishUrl   = TWiki::Func::getPluginPreferencesValue( "PUBLISHURL" ) || "";

     $initialized = 1;
 }
@@ -158,7 +159,7 @@
         $error = "Error: Can't copy $from $to ($!)";
         TWiki::Func::writeWarning( "- ${pluginName}: $error\n" );
     }
-    return "/$attachPath/$file";
+    return "$publishUrl/$attachPath/$file";
 }

 # =========================
@@ -262,7 +263,7 @@
     $text =~ /(.*)/;
     $text = $1; # untaint
     if( $type eq 'url' ) {
-        $text = '/' . $text;
+        $text = $publishUrl. '/' . $text;
     } elsif( $type eq 'file' ) {
         $text = $publishDir . '/' . $text;
     } elsif( $type eq 'label' ) {

-- DanielGrana - 10 May 2006

and here a patch to make lowercase a setting, but keeping lowercase as default behaivor. Plugin setting in TWiki/PublishWebPlugin

• set KEEPCASE = 1

--- old/lib/TWiki/Plugins/PublishWebPlugin.pm       2006-02-15 17:55:40.000000000 -0300
+++ new/lib/TWiki/Plugins/PublishWebPlugin.pm 2006-05-10 05:21:55.501896750 -0300
@@ -72,6 +72,7 @@
     $excludeTopic =~ s/,\s*/\|/go;
     $excludeTopic = '(' . $excludeTopic . ')';
     $homeLabel    = TWiki::Func::getPluginPreferencesValue( "HOMELABEL" ) || "Home";
+    $keepCase     = TWiki::Func::getPluginPreferencesValue( "KEEPCASE" );
 
     $initialized = 1;
 }
@@ -257,8 +258,9 @@
     # 'url':    '/topic_name.html'
     # 'file':   '/file/path/to/topic_name.html'
     # 'label':  'Topic Name'
-    my $text = lc( $topic ) . '.html';
-    $text =~ s/[^a-z_\.]+//go;
+    my $text = $topic . '.html';
+    $text = lc( $text ) unless $keepCase;
+    $text =~ s/[^a-zA-Z0-9_\.]+//go;
     $text =~ /(.*)/;
     $text = $1; # untaint
     if( $type eq 'url' ) {

-- DanielGrana - 10 May 2006

This patch fix TOC render bug (code from PublishContrib)

--- old/lib/TWiki/Plugins/PublishWebPlugin.pm   2006-05-10 06:05:01.726585492 -0300
+++ new/lib/TWiki/Plugins/PublishWebPlugin.pm       2006-05-10 07:31:46.164543831 -0300
@@ -134,6 +136,10 @@
     $tmpl =~ s/($pubUrl)\/([^'" ]+)/&fixAndCopyAttachments($1, $2, $pubDir )/geo;
     $tmpl =~ s/<\/?(nop|noautolink)\/?>\n?//gois;
 
+    # fix links to dinamics twiki topics (from PublishContrib)
+    my $ilt = '(?:http://localhost)?' . $TWiki::dispScriptUrlPath . $TWiki::dispViewPath . $TWiki::scriptSuffix;
+    $tmpl =~ s!(href=["'])$ilt/$web/(\w+)!"$1".&buildName($2,'url')!geo;
+
     my $name = buildName( $topic, 'file' );
     writeDebug( "publishTopic, saving file $name using $publishSkin skin" );
     TWiki::Func::saveFile( $name, $tmpl );

-- DanielGrana - 10 May 2006

Thanks Daniel for the fixes, I will take them into the next release.

-- PeterThoeny - 11 May 2006

The fix did not work in our setting. Reason was that $TWiki::dispScriptUrlPath . $TWiki::dispViewPath . $TWiki::scriptSuffix was empty.

To still make it run, we hacked the code as follows:

...
+    # fix links to dynamics twiki topics (from PublishContrib)
+    #my $ilt = '(?:http://localhost)?' . $TWiki::dispScriptUrlPath . $TWiki::dispViewPath . $TWiki::scriptSuffix;
+    $tmpl =~ s!(href=["'])/wiki/bin/view/$web/(\w+)!"$1".&buildName($2,'url')!geo;
...

-- WolfgangSlany - 08 Sep 2006

%PUBLISHWEB{"nicetopic"}% in the skin seems not to work with wikinames (it wrongly points to the internal wiki page). However, it works fine with Such_Page_Names

• I need to investigate. -- PeterThoeny - 22 May 2006

-- WolfgangSlany - 20 May 2006

With pattern skin, /p/pub/TWiki06x01/PatternSkin/TWiki_header.gif is used from the TWiki web when republishing all pages via the form on the TWiki.PublishWebPlugin page, however, when saving individual pages in the publish web, the /p/pub/TWiki06x01/PatternSkin/TWiki_header.gif of the Publish web is used. In the pattern skin, these two are by default different (TWiki Web with people and robot, Publish Web without).

• That was fixed in the 15 Feb 2006 version. Please use the Publish.WebPublish topic to re-publish all topics; the latest TWiki.PublishWebPlugin topic has no longer a publish form. -- PeterThoeny - 22 May 2006

-- WolfgangSlany - 20 May 2006

Typing a smiley at the end of a page without anything after it works fine in the wiki view, but is shown as :-) in the published version. Putting a space or newline behind works also for the published version.

• I do not see this on my installation. Probably caused by the skin used. Make sure that %TEXT% is on a line by itself in the skin template file. -- PeterThoeny - 22 May 2006

-- WolfgangSlany - 21 May 2006

A link such as Somecompany GmbH will be rendered OK in the internal wiki view, but in the published version there will be a question mark to edit http://www.somecompany.com/wiki/bin/edit/Publish/GmbH?topicparent=TWiki.PublishWebPlugin next to GmbH. Escaping GmbH with a leading ! solves the problem.

• Is a bug. I have not seen this becuse I desabled autolinking in my Publish web ( Set NOAUTOLINK = on in WebPreferences). -- PeterThoeny - 22 May 2006

-- WolfgangSlany - 21 May 2006

A ---+ Title line at the beginning of a page will not be translated correctly in the published version (it is OK in the wiki view). Solution: Put a line containing a single space before the ---+ Title line.

• I do not see this on my installation. Probably caused by the skin used. Make sure that %TEXT% is on a line by itself in the skin template file. -- PeterThoeny - 22 May 2006

-- WolfgangSlany - 21 May 2006

Setting DEBUG = 0 still writes debug messages into data/debug.txt (for every view at least 9 lines... why does it log "view" commands from non-publish webs at all? This seems very inefficient). Here are some sample lines from my data/debug.txt file (Main is not the Web that is published):

| 22 May 2006 - 22:45 | - PublishWebPlugin: initPlugin( Main.WebHome ) is OK
| 22 May 2006 - 22:45 | - PublishWebPlugin: commonTagsHandler( Main.WebHome )
| 22 May 2006 - 22:45 | - PublishWebPlugin: commonTagsHandler( Main.WebHome )
| 22 May 2006 - 22:45 | - PublishWebPlugin: commonTagsHandler( TWiki.WebLeftBarWebsList )
| 22 May 2006 - 22:45 | - PublishWebPlugin: commonTagsHandler( Main.WebLeftBar )
| 22 May 2006 - 22:45 | - PublishWebPlugin: commonTagsHandler( TWiki.LanguageSelector )
| 22 May 2006 - 22:45 | - PublishWebPlugin: commonTagsHandler( TWiki.WebTopBar )
| 22 May 2006 - 22:45 | - PublishWebPlugin: commonTagsHandler( TWiki.WebBottomBar )
| 22 May 2006 - 22:45 | - PublishWebPlugin: commonTagsHandler( Main.WebHome )

Any help? data/debug.txt is getting totally blown up

• Confirmed bug: Remove the hardcoded debug=1; line in initPlugin function. This Plugin runs on Cairo and Dakar; it could be made more efficient with conditional code. -- PeterThoeny - 22 May 2006

-- WolfgangSlany - 22 May 2006

Whenever a page gets published, the following lines are appended to data/warnXXXXXX.txt:

| 22 May 2006 - 22:54 | (TWiki::Plugins::PublishWebPlugin) - PublishWebPlugin: Error: Can't copy /netshares/www/wikis-new/pub/TWiki/PreviewBackground/preview2bg.gif);
        }
</style>
<style /netshares/www/wikis-new/pub/Publish/htdocs/_attach/preview2bg.gif);
        }
</style>
<style (No such file or directory)

Note that the file is there and has the right permissions, though I had to copy it myself (and sic!, the last line does not have a closing >).

• Confirmed bug: The Plugin does not account for URLs in style sheets, such as style="background-image: url(%PUBURL%/TWiki/MySkin/mygrid.gif);". To fix, change a line in the publisTopic function to: (change indicated in red)
  $tmpl =~ s/($pubUrl)\/([^\)'" ]+)/&fixAndCopyAttachments($1, $2, $pubDir )/geo;
  -- PeterThoeny - 22 May 2006

-- WolfgangSlany - 22 May 2006

Please consider adding the use strict; pragma to this plugin. Its use is important to ensure the quality of TWiki plugins, enables catching certain errors, and helps to avoid unpleasant surprises. See UseStrict for more.

-- MeredithLesly - 02 Jul 2006

Concerning: Introduce subdirectories for the attachments to avoid namespace issues (but that would reveil the TWiki web structure) from top of the page. This became urgent for us as more users started to make attachments with rather generic names and unaware of attachments of others, so I had to do the following workaround:

*** 157,171 ****
      my $link = "/$attachPath/$path";
      my $file = $path;
      $file =~ s/.*\///;
      my $from = "$pubDir/$path";
!     my $to   = "$attachDir/$file";
  #    writeDebug( "fixAndCopyAttachments, copying attachment from $from to $to" );
      use File::Copy;
      unless( copy( $from, $to ) ) {
          $error = "Error: Can't copy $from $to ($!)";
          TWiki::Func::writeWarning( "- ${pluginName}: $error\n" );
      }
!     return "$publishUrl/$attachPath/$file";
  }

  # =========================
--- 158,198 ----
      my $link = "/$attachPath/$path";
      my $file = $path;
      $file =~ s/.*\///;
+     my $web = $path;
+     $web =~ s!/.*/.*!!;
+     my $topic = $path;
+     $topic =~ s!.*/(.*)/.*!$1!;
      my $from = "$pubDir/$path";
!     my $to   = "$attachDir/$path";
! #    my $to   = "$attachDir/$file";
  #    writeDebug( "fixAndCopyAttachments, copying attachment from $from to $to" );
      use File::Copy;
+     my $attachDirWeb = "$attachDir/$web";
+     if (-e $attachDirWeb ) {
+          writeDebug( "dir $attachDirWeb already exists.\n\n");
+     } else {
+          writeDebug( "dir $attachDirWeb will be created.\n\n");
+          unless( mkdir( $attachDirWeb, 0775 ) ) {
+             $error = "Error: Can't create $attachDirWeb ($!)";
+               TWiki::Func::writeWarning( "- ${pluginName}: $error\n" );
+          }
+    }
+    my $attachDirWebTopic = "$attachDirWeb/$topic";
+     if (-e $attachDirWebTopic ) {
+          writeDebug( "dir $attachDirWebTopic already exists.\n\n")
+     } else {
+          writeDebug( "dir $attachDirWebTopic will be created.\n\n");
+        unless( mkdir( $attachDirWebTopic, 0775 ) ) {
+             $error = "Error: Can't create $attachDirWebTopic ($!)";
+             TWiki::Func::writeWarning( "- ${pluginName}: $error\n" );
+         }
+     }
      unless( copy( $from, $to ) ) {
          $error = "Error: Can't copy $from $to ($!)";
          TWiki::Func::writeWarning( "- ${pluginName}: $error\n" );
      }
!     return "$publishUrl/$attachPath/$path";
! #     return "$publishUrl/$attachPath/$file";
  }

The urls look a bit weird but that's still better than having the cv of another person on one's hompage. You can see on our site http://www.ist.tugraz.at/ how it looks like.

-- WolfgangSlany - 08 Sep 2006

Wolfgang, I just browsed your site, looks nice! And thanks for the bug fixes, hope I find time soon enough to make a new release.

-- PeterThoeny - 13 Sep 2006

Forgive me for asking, but can this plugin be used to publish existing webs (and multiple webs), or only the topics in a new "Publish" web?

-- JohnDeStefano - 20 Oct 2006

You can point it to an existing web. At this time it is designed to be used in one web only; the name of that web can be configured.

-- PeterThoeny - 20 Oct 2006

Thanks; I see that now: changing the SET variable in the page changes the name of the web passed for processing. If it can ever handle multiple webs in one throw, it would be great.

-- JohnDeStefano - 20 Oct 2006

How would you solve the namespace question? Possible solution: Primary web where topic go into the htdoc root; list of other webs that go into sub-dirs below that. Would that make sense? Cross-links would need to be fixed properly.

-- PeterThoeny - 20 Oct 2006

Yeah, namespace is the big problem I am seeing as well on the publishing side with another, similar plugin: in order to get cross-Web links to work, I need to publish all Webs to a folder, then redirect any links to start from that parent folder, then back down into specific Webs. Publishing to htdocs could be one solution; what about specifying a "parent" directory, which would contain an "index" file that is nothing more than a copy of the WebHome of the Main web, with built-in logic to resolve links to the actual webs below it? Not sure if that makes sense; feel free to contact me for clarification.

-- JohnDeStefano - 23 Oct 2006

Yes, treating one topic special as the htdoc root /index.html would be an alternative solution. Although I prefer to have standard web pages in the htdoc root, such as /about.html, /services.html etc.

-- PeterThoeny - 23 Oct 2006

Understood and agreed; but then how would you attack the issue? I'm not sure how else to resolve the top-level issue in cross-Web linking, since all Webs are "flat", so to speak.

-- JohnDeStefano - 23 Oct 2006

Option 1: Treat one web as top level, and a set of other webs (on same top level on TWiki) as sub dirs under htdoc root. Links between webs need to be fixed properly when publishing content. Example structure:

TWiki web structure	Htdoc structure
• Publish • Marketing • Products	/*.html /marketing/*.html /products/*.html

Option 2: One TWiki web for publish, with sub-webs as needed. Example structure:

TWiki web structure	Htdoc structure
• Publish   • Marketing   • Products	/*.html /marketing/*.html /products/*.html

Option 2 is more intuitive, but does not allow the re-use of existing webs.

-- PeterThoeny - 23 Oct 2006

Obviously, if I were egocentric, I would choose Option 1, since I have existing webs to publish! However, I think Option 2 makes the most sense and better reflects the structure of an actual, usable heirarchy.

-- JohnDeStefano - 24 Oct 2006

Would it be possible to see any approved code (or cumulative patches) that would enable the plugin to publish without having it convert the web and topic names to lowercase? I see discussion and some code about this earlier in this topic, but not sure what's safe or optimal to implement.

-- JohnDeStefano - 28 Nov 2006

Personally I do not have a need for this, but it could certainly be implemented (with a setting to disable the lowercase conversion.) We can add this if someone brings a patch that works on Cairo and TWiki 4 (including documentation.)

-- PeterThoeny - 29 Nov 2006

We're looking at using TWiki internally to develop our procedure manual. For this reason, it would be great that this plugin be interfaced with WorkflowPlugin so that we can publish pages only once they have been approved. We need to have two versions of our internal procedures, one which is under development (on the wiki server) and one which has been approved and currently under application (maybe on a different wiki web).

-- DenisBallant - 07 Mar 2007

For a page using ImagePlugin, I get different results depending on whether the static page is published during a save cycle (which works) or the publish web cycle. The publish web cycle does not have the linked in style sheet for the ImagePlugin which would indicate to me (a novice) that perhaps the ImagePlugin did not execute it's initPlugin. Any suggestions?

-- RichardBeaver - 06 Dec 2007

It may be a hack, but I was able to solve my problem by making the following changes to the ImagePlugin commonTagsHandler (lines with RB):

###############################################################################
# only used to insert the link style
sub commonTagsHandler {
# RB  return if $doneHeader;
  return if ($_[0] =~ /<head>[^\n]*$imgStyle/ );     # RB

  if ($_[0] =~ s/<head>(.*?[\r\n]+)/<head>$1$imgStyle\n/o) {
    $doneHeader = 1;
  }
}

###############################################################################

-- RichardBeaver - 07 Dec 2007

Can we please have the settings in a spec file instead of in the pm file? The current way will loose all configurations with a plugin update.

-- ArthurClemens - 21 Feb 2008

Doc update needed: Search engines prefer dashes over underscores, http://www.mattcutts.com/blog/dashes-vs-underscores/ - recommend to create topic names like About-us instead of Acout_us.

-- PeterThoeny - 2010-10-05

It would be nice if the user could configure whether they want the publish to happen on every save or just on demand, as in the PublishContrib.

An extensive generation of content from a dynamic topic could be time consuming and not be desirable during the edit/save cycle.

Further, one might not want to publish everytime one makes a change, depending on the purpose of the published site.

-- ThomasWeigert - 2011-11-27

This could be done with a new configure setting.

On need to no publishing before content is ready, at work we actually publish first automatically to a staging area with the PublishWebPlugin, then to sync to the corporate website using rsync (used in a CGI script that is called via a browser link). This approach has the advantage that one can make changes to many topics and push out the changes after review.

-- PeterThoeny - 2011-11-27

Published file name not support UTF-8, how can i fix it ? thx!

-- LiamChou - 2011-11-29

This plugin has not been tested for UTF-8 support. TWiki is open source, where everybody is invited to take and contribute. I invite you to get involved and send a patch. Alternatively you can hire a consultant to do that for you.

-- PeterThoeny - 2011-11-29

So back in 2006 Peter asked if anyone needed support for publishing multiple webs. Well, I do, so I wrote a patch to that end. It creates a new boolean config variable, AppendWebnameToPath (defaults to off), which if set appends the web name to the publishdir and attachdir. Like always, you still need manually create those dirs and chown them to the webserver user.

--- lib/TWiki/Plugins/PublishWebPlugin/Config.spec.dist   2012-03-02 10:48:12.103829568 -0600
+++ lib/TWiki/Plugins/PublishWebPlugin/Config.spec        2012-03-01 16:52:26.683852378 -0600
@@ -30,4 +30,7 @@
 # URL path that corresponds to <code>{PublishPath}</code> directory. Leave
 # empty if it is the HTML document root.
 $TWiki::cfg{Plugins}{PublishWebPlugin}{PublishUrlPath} = '';
+# **BOOLEAN**
+# Append the webname to PublishPath and AttachPath
+$TWiki::cfg{Plugins}{PublishWebPlugin}{AppendWebnameToPath} = 0;
 1;

--- lib/TWiki/Plugins/PublishWebPlugin.pm.dist    2012-03-01 16:27:14.575903461 -0600
+++ lib/TWiki/Plugins/PublishWebPlugin.pm 2012-03-01 16:49:09.551896845 -0600
@@ -44,6 +44,7 @@
 my $publishDir;
 my $attachDir;
 my $publishUrlPath;
+my $appendWebnameToPath = 0;

 # template path for skin file; empty for twiki/templates; must be absolute path if specified
 $templatePath = $TWiki::cfg{Plugins}{PublishWebPlugin}{TemplatePath} || "";
@@ -57,6 +58,9 @@
 # URL path corresponding to $publishPath
 $publishUrlPath = $TWiki::cfg{Plugins}{PublishWebPlugin}{PublishUrlPath} || "";
 
+# if set to true, append the web name to publishPath and attachPath
+$appendWebnameToPath = $TWiki::cfg{Plugins}{PublishWebPlugin}{AppendWebnameToPath} || 0;
+
 # =========================
 sub initPlugin
 {
@@ -73,6 +77,12 @@
     # Get plugin debug flag
     $debug = TWiki::Func::getPluginPreferencesFlag( "DEBUG" );
     
+    # append web name to path?
+    if( $appendWebnameToPath == 1 ) {
+        $publishPath = $publishPath . '/' . $web;
+        $attachPath = $attachPath . '/' . $web;
+    }
+
     writeDebug( "initPlugin( $web.$topic ) is OK" );
     $initialized = 0;
     $error = "";

I'd love to see this pulled back into the main plugin, if there is anything I can do to improve the odds of that let me know. Thanks.

-- JohnVestrum - 2012-03-02

Thank you John for supporting the TWiki open source project! This is a good enhancement. I'll take that into the next release. I'd like to make two small changes. Explained and tracked at TWikibug:Item6856 - please follow-up there.

-- PeterThoeny - 2012-03-05

I implemented John's patch with some changes. Instead of a new {AppendWebnameToPath} setting, all path settings now accept these new variables:

• %WEB% - name of publish web
• %LCWEB% - name of publish web, lower case version
• %SKIN% - name of publish skin

This should cover John's needs, and adds the flexibility to publish content to multiple virtual hosts. Example, using web PublishXYZ and skin xyz.com as an example:

• Setting in WebPreferences of PublishXYZ:
  • Set PUBLISHWEBPLUGIN_PUBLISHSKIN = xyz.com

• Configure settings:
  • $TWiki::cfg{Plugins}{PublishWebPlugin}{PublishPath} = "/var/www/vhosts/%SKIN%/html";
  • $TWiki::cfg{Plugins}{PublishWebPlugin}{TemplatePath} = '/var/www/vhosts/%SKIN%/html/_skin';

The skin name xyz.com is now used by the publish process to point the proper virtual host at /var/www/vhosts/xyz.com. The skin file is located at /var/www/vhosts/xyz.com/html/_skin/xyz.com.html.

-- PeterThoeny - 2012-03-05

I made a couple changes to my copy of the plugin, to support (1) https links and (2) anchor links (mypage#myparagraph) within double square brackets. Here's the diff, in case there is any interest in pulling it back in.

--- PublishWebPlugin.pm 2012-06-12 15:40:26.000000000 -0500
+++ /home/msi/vestrum/PublishWebPlugin.pm       2012-10-04 09:37:16.234280000 -0500
@@ -259,7 +259,7 @@
 sub handleLink
 {
     my ( $link, $label ) = @_;
-    if( $link =~ /^(http|ftp)\:/ ) {
+    if( $link =~ /^(http|https|ftp)\:/ ) {
         return "<a href=\"$link\">$label</a>";
     } elsif( $link eq $label ) {
         return '<a href="'
@@ -358,11 +358,16 @@
     # 'label':  'Topic Name'
     # 'publishurlpath': {Plugins}{PublishWebPlugin}{PublishUrlPath} configure setting
     my $text = lc( $topic ) . '.html';
-    $text =~ s/[^a-z0-9_\-\.]+//go;
+    $text =~ s/[^a-z0-9_\#\-\.]+//go;
     $text =~ /(.*)/;
     $text = $1; # untaint
     if( $type eq 'url' ) {
-        # keep text as is (relative URL)
+        # handle anchors
+        if( $topic =~ /\#/ ) {
+            my $anchor = lc( $topic );
+            $anchor =~ /^([^\#]+)(\#.*)$/;
+            $text = $1 . '.html' . $2;
+        }
     } elsif( $type eq 'file' ) {
         $text = $publishDir . '/' . $text;
     } elsif( $type eq 'label' ) {

-- JohnVestrum - 2012-10-04

I'm glad to see activity in this thread again, after a haitus of several years, and encouraged at the improvements being made. But I'm a bit confused by the motivation: I'm already publishing multiple webs using the prior version of the plugin, albeit with a small bash script and some Apache regexp hacks. Perhaps this new version obviates the need to manipulate Apache requests to published webs?

-- JohnDeStefano - 2012-10-05

Comments: How to uninstall an existing plugin from TWiki Server ?

We have installed this PublishWebPlugin but unable to publish now using the same. Error when trying to publish is: " TWiki detected an internal error - please check your TWiki logs and webserver logs for more information. Can't create file path - No such file or directory" So we are trying to uninstall this PublishWebPlugin now.

-- NivethithaAnnamalainathan - 2012-11-23

Nivethitha, read the TWikiPlugins documentation on how to enable/disable plugins.

-- PeterThoeny - 2012-11-24

We installed PublishWebPlugin in our server and also we were able to publish the contents as HTML pages before but now we are unable to publish. Getting the following error " TWiki detected an internal error - please check your TWiki logs and webserver logs for more information. Can't create file path - No such file or directory". We dont want to disable the plugin from configure page, we have to uninstall the plugin from the server by removing all the PublishWebPlugin related files from the backend.

-- NivethithaAnnamalainathan - 2012-11-27

Nivethitha, please take the time to RTFM. Disabling the plugin will solve the issue. If you want to remove the files look at the file list of PublishWebPlugin.zip indicated in PublishWebPlugin and remove those files in your TWiki installation.

-- PeterThoeny - 2012-11-28

Hi, I'm trying to use the HistoryPlugin with the PublishWebPlugin but the publish activity is ignoring the %HISTORY% variable. Is this a quick fix? Thanks.

-- Sally Houghton - 2014-05-01

Sally, I do not know, I have not used the HistoryPlugin. I just checked the code, the plugin uses the register tag handler for the %HISTORY% variable, so it should get expanded in the PublishWebPlugin, but there is also a template file, and I am not sure about that.

-- Peter Thoeny - 2014-05-01

I've rewritten handleLink to address the issue with anchor tags, See Bugs web Item6968. This change seems to fix the problem...

sub handleLink
{
    my ( $link, $label ) = @_;

    if( $link =~ /^(https?|ftp)\:/ ) {
        return "<a href=\"$link\">$label</a>";
    }

    my $url = '';
    my ($name, $anchor) = split(/#/,$link);
    if ($name ne '') {
        $url = buildName($name,'url');
    }
    if ($anchor ne '') {
        $url .= "#$anchor";
    }

    if( $link eq $label ) {
        return '<a href="' . $url . '">' . buildName( $link, 'label' ) . '</a>';
    } else {
        return "<a href=\"$url\">$label</a>";
    }
}

-- Michael Sprague - 2014-10-26

Thank you Michael! This is now in SVN trunk & 6.0 branch, and new plugin package is released as well.

-- Peter Thoeny - 2014-10-26

It seems not all local links are being handled properly. I noticed this by trying to use TOC{"OtherTopic"}. It seems to to be fixed if you replace:

    $tmpl =~ s/( href=['"])$viewWebUrl([^'"])*/$1 . buildName( $2, 'url' )/ge;

in publishTopic with:

    my $viewPathUrl = $viewWebUrl;
    $viewPathUrl =~ s/http[s]?:\/\/[^\/]*//;
    $tmpl =~ s/<a\s*?href\s*?=\s*?['"].*?$viewPathUrl([^'"]*).*?>(.*?)<\s*?\/a>/handleLink($1,$2)/ige;

Also, the following code to remove the URL parameters needs to be moved up to execute before this section (which is commented as "fix page links"):

    # remove URL parameters to make TOC and other TWiki internal links work
    $tmpl =~ s/(<a href=[\"\'][A-Za-z0-9_\-\/]*)\?[^\#\"\']*/$1/gos;

-- Michael Sprague - 2014-11-23

Thank you Michael! This is tracked at TWikibug:Item7593.

-- Peter Thoeny - 2014-12-04

New release done with Michael's fix.

-- Peter Thoeny - 2014-12-05