TWiki2GoUpdate

Log of last update

last update done 2007/11/28 16:37:

 20535 files scanned
    35 TWiki webs found
  4581 TWiki topics found
     0 TWiki topics copied
    15 TWiki topics not copied, because they are bigger than 50000 bytes
  4070 files found in ./twiki/pub/
     0 attachments copied
  1595 attachments not copied, because they are bigger than 50000 bytes
     0 errors ocured

D:/TWiki_offline/TWiki2Go/htdocs/twiki/bin/zz_update_twiki2go.pl finished in 3 seconds

PS: this does not work with TWiki.org, the button above is only an example, how this would look like in your local TWiki, updating from your TWiki server.

Description

1. Scan all <web>/WebPreferences for web wide access rights
2. Scan all webs to find all topics (i.e. *.txt files) that are not protected by TWikiAccessControl
3. Update topics in destination
4. Scan all attachments
5. Update all attachments (that are attached to topics which are not protected by TWikiAccessControl)

Usage

1. Use TWiki2GoUpdateContrib to copy TWiki topics and attachments to another directory on your server. This can be done regularly with a cron-job.
2. Create a Windows share on the server, the Windows clients have to map this share
3. The client user can update manually with the 'update TWiki2Go' button above.

Command line parameters

TODOs

• Improve script to take over URL parameters -> mostly done
• Different log output when started from command line. -> done
• Write log file to disk.

Related topics

• Plugins.PublishContrib: Add-on to create static HTML topics.

-- Contributors: MatthiasThullner - 27 Dec 2007

Discussion

Syncing TWikis is described in "ReadWriteOfflinePlugin enables roaming TWiki users", a plugin by CDot.

-- MichaelDaum - 27 Dec 2007

Michael, thanx for the comment, interesting article.

Will the ReadWriteOfflinePlugin be published on TWiki/available for the TWikiCommunity?

I do not want to sync, but allow the users to take our TWiki knowledge database with them when they are on the go, so the update is mostly a performance issue.

-- MatthiasThullner - 28 Dec 2007

You will have to talk to CDot.

If you don't want to sync, do you strip off a read-only copy of the knowledge database? You won't need more than static HTML in that case. However, if you are in "danger" of having content being modified on both/multiple ends, your sync situation is potentially quite complex.

-- MichaelDaum - 28 Dec 2007

I tried PublishContrib, which worked fine, but I missed some key features:

• Search did not work.
• I like the idea to use TWiki2Go as a notpad, i.e. having an additional 'local user web' (did not think about syncing the 'local user web').
• PublishContrib could not handle this:
  • We are having lots of topics with same names (only case sensitive), redirecting to each other. There was no control which one would 'survive' on Windows.
  • Update will be difficult, as all topics will be written newly and it is not easy to find out which topics have to be updated locally, specially for the topics containg dynamic content like %SEARCH...
  • Speed. Puplishing took hours :-(. We are having about 6000 topics and 12000 attachments, see TWiki2GoUpdate log:

     56198 files scanned
        35 TWiki webs found
      6558 TWiki topics found
      6506 TWiki topics copied
        52 TWiki topics not copied, because they are bigger than 50000 bytes
     30666 files found in ./twiki/pub/
     12235 attachments copied
      6050 attachments not copied, because they are bigger than 50000 bytes
         0 errors ocured 

  • We are also referencing a mirror of our configuration management system in TWiki, which should be made available locally also.
• Our intranet TWiki is running TWiki 3, so I had to update the content to another newer TWiki anyway to use PublishContrib. Also I do not want to cause high load on the server.

You brought the syncing to my mind, I think I will update this feature for some webs (non local user webs) later on. As we have about 500 users, I think a web for each user on the server will be too much.

-- MatthiasThullner - 28 Dec 2007

Oh, and if you don't need merging on the file-level, unison is another option. Great tool, I use it daily.

-- MichaelDaum - 28 Dec 2007

improved script zz_update_twiki2go.pl and ./lib/TWiki/Contrib/TWiki2GoUpdate.pm

Tested only a bit on

• Web interface on TWiki 3 and 4.1.2 on Linux server
• Command line and Web interface on TWiki 4.2rc2 on Server2Go on Windows

-- MatthiasThullner - 28 Dec 2007

Matthias, I developed ReadWriteofflinePlugin at my own cost in anticipation of sponsorship from a company who were interested in the tech. They didn't go ahead, and a couple of other subsequent attempts to get sponsorship have also gone by the wayside. The ReadWriteOfflinePlugin is a generic peer-peer synch mechanism which allows synching of individual webs, including merging of edits on different sites. The only limitation is that histories are site-specific, rather than synched (this is unlikely to change unless I get sponsorship, in which case I would release open-source as well).

If you have requirements for PublishContrib please go ahead and add them to TWikibug:PublishContrib. Maybe someday I'll get some support to implement them.

-- CrawfordCurrie - 29 Dec 2007

Thanks Matthias for contributing this add-on. Could you please create a TWiki2GoUpdateAddOn in the Plugins web and move the content there? I suggest to package this add-on as TWiki2GoUpdateAddOn.zip. See AddOnPackageHowTo.

-- PeterThoeny - 02 Jan 2008