Question

I am interested in usening a remembrence agent for indexing / searching my TWiki, as I only have gross ksh scripting skills I really have no idea on where to start to tie RA to a TWiki.

Here is the Remembrance Agents Page.

I have this (interesting) program compiled and working (basicly) in command mode. What I would like help in doing is makeing the results of this tool availible on demand (dynamic per page). Margin Notes for TWiki is my visualised goal.

What I can do so far:

• I can create index files per web

• I can get results from the search interface:

# (echo query; cat search.txt) | $TWIKIHOME/RA/bin/ra-retrieve $TWIKIHOME/RA/data/Know 
1   0.50 | 1 | |||||KnowledgeBaseSpec.txt|nobody||||||||||||||||||KnowledgeBaseSpec.txt|users, file, version, current, source | 0.50, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00
2   0.20 | 25 | %META:TOPICINFO{auth|||||WebPreferences.txt|nobody||||||||||||||||||WebPreferences.txt|users | 0.20, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00
3   0.17 | 0 | %META:TOPICINFO{auth|||||IncorrectDllVersionW|nobody||||||||||||||||||IncorrectDllVersionW32PTH10DLL.txt|file, version | 0.17, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00
 Various|||||ProblemPrinting.txt|nobody||||||||||||||||||ProblemPrinting.txt|file, details | 0.11, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00
5   0.07 | 14 | %META:TOPICINFO{auth|||||ReadmeFirst.txt|nobody||||||||||||||||||ReadmeFirst.txt|details, version | 0.07, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00
.
Unknown command, type ? for help.

#

Anyone?

• TWiki version: 01 Dec 2001
• Web server: Apache
• Server OS: FreeBSD
• Web browser: Opera
• Client OS: XP

-- DrewCard - 05 Mar 2002

Answer

This looks very interesting - since RA runs continuously it's really an additional search feature rather than an alternative, I think. The UI presumably should present the Margin Notes in a frame or using tables/CSS to position them on the side somewhere (see the various CSS discussions on TWiki's Codev web) - you might also like to look at MozillaSidebar, this is what Netscape use for their somewhat similar 'what's related' feature.

The implementation of the Margin Notes feature is somewhat driven by the chosen UI - however, if you implement a plugin for TWikiVariables, e.g. providing %RASEARCH{....}% to display the output of RA in a given place, you could make the UI quite independent of the implementation. This is a bit like FormattedSearch (used by TWikiSyndication and MozillaSidebar). The main issue, if you use a separate frame/window of some sort, is telling the RA frame when the user moves to a different TWiki topic in the main window - not sure how you would do this without some JavaScript on every TWiki link (which is feasible but not very nice...). If you embed the RA search results in a table or CSS area on the same page, it's a lot easier. Either way, a Plugin is the way to go for most of the code - even if the core code must be updated slightly it does limit changes to the core.

RA would also be useful as a variant of the SuggestLinksPlugin, since associated docs found by RA are good candidates for linking from a page.

Some other search-related pages are WhatAboutSearchEngines and these pages with 'Search' in their name. You might also want to look at XmlRpcForWebApps - this mentions a nascent XML-RPC interface to TWiki, UseModWiki and other Wikis, which LesOrchard is working on, which might be an interesting option in the longer term. Alternatively, have a look at RichSiteSummary, though that's probably less relevant since RSS feeds are implicitly time-ordered.

One minor point - you may want to implement RefreshEditPage since you are using Opera. This avoids possible loss of edits when re-editing a page.

Can someone move this topic into Codev, it will simplify linking to Codev stuff and really belongs there?

-- RichardDonkin - 06 Mar 2002

Okay, not sure about the Codev stuff. Pulled the RefreshEditPage fix onto my to-do list.

Lots of good ideas. Thinking about it now it seems the best if something could be done that gives static results inthe footer to the page, simulateing/emulateing the emacs frame/window. This should result in a simpler implementation. Okay I can see this working as an add on/in. Now who can I talk into helping me make RASearchPlugin a reality?

-- DrewCard - 11 Mar 2002

Assuming you are OK with Perl coding, have a look at some of the plugins in the Plugins web - they should give you an idea of how to code things, and of course see the TWikiDocumentation. Feel free to comment here or in Codev.

It would be particularly useful if RA could work on the preview text of a page, or via a special button on the Edit page, i.e. to analyse pages related to the text as it is entered. This could help avoid duplicate questions on busy sites such as TWiki.org, by showing pages that ask and answer the same questions - sort of a dynamic FAQ system without having to create a special structure!

-- RichardDonkin - 13 Mar 2002

Have a look at SherlockAndMozillaSearching - with Mozilla at least (and perhaps InternetExplorer, see MozillaSidebar), it would be very easy to keep the sidebar frame updated all the time, simply by generating suitable search results from RA in every page. (This is what Google does if you enable the MozillaBrowser search sidebar - even a search in the main page updates the sidebar search results.)

If you put the RA variable into a template, and hide it within an HTML comment, you could automagically enable a whole TWiki site for RA, given a suitable plugin. MikeMaurer is working on sidebars for IE6; for other browsers you could use a template that shows the RA results as part of the main page, so the whole thing can be cross-browser while still using sidebars where they are available.

-- RichardDonkin - 15 Mar 2002