Mass-update of topics with a global search and replace

RenameTopic capability has been pretty popular, although it's not without its problems. A lot of the code in it is really above changing text, in this case references, in a lot of topics. RenameTopic includes the ability to:

• Display all topics that could be changed
• Show snippets of text in topic to be changed
• Indicate and avoid changing locked topics
• Allow you to change locked topic later

This is a lot of what you need for doing an update of text in many topics. e.g. say you want to change the name of a form field, but of course it's present in many topics.

The sort of thing I'm thinking of would allow a regular expression based replace, probably including the ability to specify context like the substitution operator in Perl.

This is not something you'd use every day, but it would be a useful tool when used carefully. Any interest? Anyone like to help enhance the rename code so it can also do this?

-- JohnTalintyre - 10 Sep 2003

Re: Any interest? Yes — I've wished for this myself at times, and will need it (almost certainly) when I migrate WikiLearn to it's own web site. (Of course, the alternative is to edit the .txt files outside TWiki, probably at the cost of losing the RCS record.)

Re: Anyone like to help enhance the rename code so it can also do this? Sorry, a passing interest, but I don't really even know Perl. (and am currently focused on learning Python (and C)).

-- RandyKramer - 10 Sep 2003

There is a certain TWikiGuest who desperately needs this now!

-- ArthurClemens - 12 Sep 2003

Yes, Michael needs this feature desperately, indeed!

Actually, a mass update feature would be a very useful feature at my daytime workplace. We do have large scale refactoring, now with over 30K topics is difficult to impossible to do that by hand.

I suggest to separate 1. search, 2. find, 3. select, and 4. replace. That is:

1. Search for topics (existing feature, search by topic or text)
2. Find occurances of text, multiple per topic, regex (not necessarily the same text as search)
3. Select found text, multiple per topic (select all, clear all)
4. Replace found and selected text with new text

Example:

1. Search for topics starting with "MeetingMinutes", e.g. a regex topic search for ^MeetingMinutes
2. Find occurances of %INCLUDE{".*"}%
3. Leave all as select (default)
4. Replace found and selected text with %INCLUDE{".*" warn="on"}%

This can be a dangerous feature and needs to be access controlled.

-- PeterThoeny - 14 Sep 2003

At work we need this feature. It will be released probably as a Plugin. Stay tuned.

The implementation is actually not too difficult and can be done without changing the core code. The idea is to generate a FormattedSearch where the result contains a Plugin specific variable. The Plugin expands that with checkboxes and near by text.

-- PeterThoeny - 23 Dec 2003

I have come across a tool that achieves about everything discussed here, although it doesn't do it from within TWiki. It's a script called Fileman that offers browser-based ftp capabilities plus some other very useful features for TWiki installations on hosted domains. The single-user version is free. It includes a search and replace feature that works both on filenames and file content and also allows regex. I have used it to change topic names and also variable names within topics. Very useful!

-- LynnwoodBrown - 21 Jan 2004

here's a small script to do a global search and replace:

#! /usr/bin/perl -w
use strict;
use XMLRPC::Lite;

my $login = { username => 'TWikiGuest', password => 'TWikiGuest' };
my $proxy = XMLRPC::Lite->proxy( 'http://sane-asylum.com/twiki/bin/xmlrpc-voodoo' );

my @pages = @{ $proxy->call( 'vpwiki.getPageKeys', $login )->result };
foreach my $page ( @pages )
{
    print STDERR "\n$page ";

    my $data = $proxy->call( 'vpwiki.getPage', $login, $page )->result;

    next unless (my $pageStruct = $data)->{page};
    my $origPage = $pageStruct->{page};
    $pageStruct->{page} =~ s|$search|$replace|g;

    # don't bother writing if we haven't changed anything (slower and creates an unnecessary revision)
    next if $origPage eq $pageStruct->{page};
    print STDERR "changed"
        if $proxy->call( 'vpwiki.setPage', $login, $pageStruct )->result;
}

and, it's pretty easy to change it from global search and replace to a more selective one:

my @pages = grep { /search-criteria/ } @{ $proxy->call( 'vpwiki.getPageKeys', $login )->result };

-- WillNorris - 21 Jan 2004

I'm afraid I couldn't make complete sense of the above code. As said at the top of the topic, the code for this (and more) already exists in RenameTopic. Order or things being:

• User supplies string to match
• User supplies string to replace
• All Webs or just current
• Show matching topics, highlight changes to be made
  • Indicate if lock on topic
• All matches to be selected, or user selects topics individually
• Make changes
• Show topics that remain because of locks

-- JohnTalintyre - 24 Jan 2004

OK, the first version of the Plugins.GlobalReplacePlugin is now released. Enjoy

For security reasons I do not intend to install this Plugin at TWiki.org.

-- PeterThoeny - 08 Feb 2004