Bug: Backlink search does not work

There are several problems in the current TWikiRelease04Sep2004 with the Backlinks link in the navigation bar below.

1. Backlinks search fails to find:
   • WikiWord if it is the last word on a page or it is followed by newline (\n or \r).
   • a plural WikiWord that links to a singular topic name (i.e. on Wikis with $doPluralToSingular enabled in lib/TWiki.cfg).
2. Backlinks search finds spaced out words although they do not link to the topic name (as they are not written inside brackets).
3. Backlink search doesn't work across webs.

Test Case

Test for a) - b):

Go to BacklinksSearchTest. That topic contains links to several other topics which link back using different methods.

Now click the Backlinks link in the navigation bar there. The results should contain all topics listed on BacklinksSearchTest except the ones beginning with BackLinkNoReference...

Test for c):

Click on Backlinks below. You will get results from codev web only (although Sandbox links here, too).

Environment

-- DavidForrest - 28 Sep 2004

Follow up

It also doesn't work across webs, but I hadn't noticed about the lack of whitespace problem you mention.

-- MartinCleaver - 28 Sep 2004

Yes. The not working across webs is confusing our workgroup because we stupidly have extra default webs with meaningless and probably default names. Categorization using 'CategoryCategory' type tags is awkward when relying on the unreliable search tool.

-- DavidForrest - 30 Sep 2004

Fix record

the problem occurs also if the pattern being searched is followed by a \n. the problem is in the way the Backlinks regular expression is built (the string is matched as long as it is not followed by a valid WikiWord character). a more general way to match the end of a word could be \b. the problem is not in the software, it is in the template you're using. you could also try to match ([^a-zA-Z0-9]|$), but \b is a lot shorter...

in the Backlink anchor of your template replace the search=%SPACEDTOPIC%%5B%5EA-Za-z0-9%5D" with search=%SPACEDTOPIC%%5Cb"

-- MarioFrasca - 04 Mar 2005

someone write enabled on SVN please commit this (view.pattern.tmpl in template/ and template/Main/ dirs).
thanks.

-- MarioFrasca - 15 Mar 2005

I experienced this problem. Just fixed it myself, but the solution was already here ...

-- PeterHuisken - 23 Mar 2005

Btw, the problem also occurs, when the WikiWord is followed by EOL. Above workaround work for this as well. It would be nice to have backlinks in the current web, as well as backlinks in all webs. I am about to put this in my working installation.

-- PeterHuisken - 23 Mar 2005

Let us know how you implement this!

-- ArthurClemens - 23 Mar 2005

I made the topic title, next to edit/attach/print BacklinksSearchDoesNotWork) clickable, so that it is easier to use than the Backlinks in the bottombar. I added All Backlinks, it lists backlinks from all webs. I only changed view.pattern.tmpl.txt. Btw, TWiki is real cool. I use SpeedyCGI to get good performance.

-- PeterHuisken - 24 Mar 2005

Hmm, lets just say, it is work-in-progress.

The "All backlinks" does not find the backlinks in the current web. Don't know to fix this..., which may be the reason you asked. For now, I use backlinks a lot for finding people's contribution, and these are always %WEB%.TWikiGuest

-- PeterHuisken - 24 Mar 2005

For me your solution works allright. Together with Mario's suggestion, the search becomes search=Codev.BacklinksSearchDoesNotWork(%24%7C%5Cb).

Why would it be necessary to keep a separate (Web) "Backlinks"? With "All backlinks" (which can then just be the default "Backlinks") you will get a stratification of the search results, grouped per web.

-- ArthurClemens - 24 Mar 2005

I would keep "Backlinks" and "All backlinks", for the following reasons:

• Let user/groups have their own Wiki Web inside a TWiki site. They want to use backlinks inside the current web, without using the long form, with the web name in front.
• References in the form [[Wiki Word]] are not found.
• References in the form WikiWord (without the Web. in front of it) do not work
• Performance. No fun searching all the webs, if you're interested in one.

Therefore, I would suggest to have it this way:

• Keep the current extended form of "Backlinks" in the current web, working with sort of links, inside the current web.
• Have a "Backlinks from all/other webs". Preferably, this should also yield backlinks in the current web. Possibly, this is doable with a nested search (don't know TWiki well enough yet). If not, I'd settle with a "Backlinks from other webs".

-- PeterHuisken - 25 Mar 2005

You have a point. In that case I would use the link labels:

Backlinks: Web All

-- ArthurClemens - 25 Mar 2005

New code:

Backlinks: <a href="%SCRIPTURLPATH%/search%SCRIPTSUFFIX%/%WEB%/?scope=text&amp;regex=on&amp;excludetopic=%TOPIC%&amp;search=%WEB%.%TOPIC%(%24%7C%5Cb)">Web</a> <a href="%SCRIPTURLPATH%/search%SCRIPTSUFFIX%/%WEB%/?web=all&amp;scope=text&amp;regex=on&amp;excludetopic=%TOPIC%&amp;search=%WEB%.%TOPIC%(%24%7C%5Cb)">All</a>

This is now in SVN (3887).

-- ArthurClemens - 25 Mar 2005

That code still failed for me. It only works if you use the %WEB%. prefix in front of the topic name in references from the same web. This is working for me:

Backlinks: <a href="%SCRIPTURLPATH%/search%SCRIPTSUFFIX%/%WEB%/?scope=text&amp;regex=on&amp;excludetopic=%TOPIC%&amp;search=(%5E%7C%5Cb%7C%WEB%.)%TOPIC%(%24%7C%5Cb)">Web</a> <a href="%SCRIPTURLPATH%/search%SCRIPTSUFFIX%/%WEB%/?web=all&amp;scope=text&amp;regex=on&amp;excludetopic=%TOPIC%&amp;search=%WEB%.%TOPIC%(%24%7C%5Cb)">All</a>

-- AdamFanello - 30 Mar 2005

Interesting; you are not using SPACEDTOPIC, that was explictly implemented to support backlinks. Does that mean we can get rid of SPACEDTOPIC?

-- CrawfordCurrie - 30 Mar 2005

I do not recommend to retire SPACEDTOPIC because this would break the majority of the 29 SkinPackages. In addition, we would not be able to detect [[Foo bar]] type links pointing to FooBar topic.

-- PeterThoeny - 30 Mar 2005

Well the new backlinks search string still did not work. It appeared that putting in %WEB% was not right. The new search:

Backlinks: <a href="%SCRIPTURLPATH%/search%SCRIPTSUFFIX%/%WEB%/?scope=text&amp;regex=on&amp;excludetopic=%TOPIC%&amp;search=%TOPIC%(%24%7C%5Cb)">Web</a> <a href="%SCRIPTURLPATH%/search%SCRIPTSUFFIX%/%WEB%/?web=all&amp;scope=text&amp;regex=on&amp;excludetopic=%TOPIC%&amp;search=%TOPIC%(%24%7C%5Cb)">All</a>

Now tested and in SVN (3939).

-- ArthurClemens - 30 Mar 2005

Just checking for \b and end of text does not account for border cases:

• Topic ending in alpha followed by a numeral, e.g. backlink search for FooBar topic should exclude FooBar2 topic
• Topic ending in a numeral followed by alpha, e.g. backlink search for FooBar2 topic should exclude FooBar2more topic
• Spaced links are not detected, e.g. backlink search for FooBar topic should account for [[Foo bar]] link.

I tested this to some degrees, this should support the border cases:

search=%SPACEDTOPIC%%28%5B%5EA-Za-z0-9%5D%7C%24%29

which searches for the spaced topic, followed by ([^A-Za-z0-9]|$)

This should be fixed in the ClassicSkin as well. And also the links in the "More" screen.

-- PeterThoeny - 30 Mar 2005

I would prefer to have both, backlinks and backlinks from other webs. Backlinks from other webs always search for the full, non-spaced Web.Topic, and backlinks searches for spaced links as well.

Not that I am a 100% purist, but it would be nice to have the backlinks button(s) easier accessible, next to edit/attach/print as well. (I am still novice w.r.t. to TWiki internals. And my God, do I hate perl...)

-- PeterHuisken - 31 Mar 2005

I've updated the templates in SVN 3953, and tested for above border cases. The classic view template is also updated, as well as the oopsmore template.

Finally, the search should include %WEB%., so: search=%WEB%.%SPACEDTOPIC%%28%5B%5EA-Za-z0-9%5D%7C%24%29

-- ArthurClemens - 01 Apr 2005

Arthur, this does not work. Backlinks should not include %WEB%. References inside the current do never have %WEB% in front. Please use this for Backlinks:

search=%SPACEDTOPIC%%28%5B%5EA-Za-z0-9%5D%7C%24%29

Now, I have all backlinks working as well. Here it is:

search=%SPACEDTOPIC%(%24%7C%5B%5EA-Za-z0-9%5D)|%WEB%.%TOPIC%(%24%7C%5B%5EA-Za-z0-9%5D)

This is working very nicely at our internal site. Btw, we finally have our philips look and feel running, found some other TWiki bugs during this. For example, the EDIT button refers to %TOPIC%, it should refer to %BASETOPIC%. Will report this rsn, with fix.

-- PeterHuisken - 15 Apr 2005

You're right. Made a fix (again) to the templates in SVN.

I have also changed the home link of the breadcrumb to

<a href="%SCRIPTURLPATH%/view%SCRIPTSUFFIX%/%WEB%/%HOMETOPIC%">%WEB%</a>

But the All link does not work yet as expected: because for other webs one needs to search with %WEB%.%TOPIC%, the links in the local web are missed. So in fact we would need 2 searches: one for the local web and one for the other webs. This cannot be done with one search link, so I think we need a Backlink template for this.

Actually I think we would need 2 (small) templates to account for both the web search and the all search. This idea continued in BacklinksTemplate

For now I've changed the link label "All" to "Other webs".

I've marked this topic as UnderConstruction again.

-- ArthurClemens - 15 Apr 2005

Arthur, the All Backlinks from my previous reply does work perfectly. I used the regex "|" operator to get both global and local links working..... And we have it running, it works.

Note: I guess I made a type in above All backlinks search strings, somehow the closing bracket was missing. I cannot check correct operation right now.

-- PeterHuisken - 16 Apr 2005

See my remark in BacklinksTemplate.

-- ArthurClemens - 16 Apr 2005

arthur, i notice you've checked in changes to SVN. is this now fixed?

-- WillNorris - 24 Apr 2005

I have udated the search expression, but we still need to do the BacklinksTemplate.

-- ArthurClemens - 24 Apr 2005

BacklinksTemplate has been finished.

There is no final solution, other than using 2 specific templates, one for "search web" and one for "search all webs". See BacklinksTemplate that is included in DakarRelease.

-- ArthurClemens - 10 Jan 2006

I take it this report covers two problems: a) crafting a correct regular search expression to find all occurences of a topic name used as Wiki link b) searching for backlinks in the current web and in other webs. BacklinksTemplate is the solution for both problems.

I'm running TWikiRelease04Sep2004. Consequently I'm trying to fix my system (this is problem a) only). Let's call it fix for TWikiRelease04Sep2004. It tries to answer the question what search string I have to enter in view.pattern.tmpl to get backlink search working for current web only. The fix record offers and discards so many different search strings that the fix is not clear to me.

-- DanielKabs - 10 Jan 2006

OK. I have this in the "backlinks web" template:

%SEARCH{ search="%TOPIC%([^A-Za-z0-9]|$)|%SPACEOUT{"%TOPIC%" separator=" *"}%([^A-Za-z0-9]|$)|%WEB%.%TOPIC%([^A-Za-z0-9]|$)" type="regex" scope="text" nosearch="on" excludetopic="%TOPIC%" web="%WEB%" }%

-- ArthurClemens - 10 Jan 2006

Arthur, your template for Dakar contains Backlinks *Search *Does *Not *Work. If this is evaluates to the same value as %SPACEDTOPIC% = Backlinks%20*Search%20*Does%20*Not%20*Work the search string is similar to the one suggested by PeterHuisken. The only difference is the hex encoding of special characters (%24 and so on).

I think the second part of the regex %WEB%.%TOPIC%([^A-Za-z0-9]|$) is redundant as the first part already matches %TOPIC%([^A-Za-z0-9]|$).

-- DanielKabs - 12 Jan 2006

Renamed from to SearchDoesNotWork to BacklinksSearchDoesNotWork

If I remember correctly you need %WEB%.%TOPIC%([^A-Za-z0-9]|$) for the Main.WebHome syntax.

-- ArthurClemens - 12 Jan 2006

I think, %WEB%.%TOPIC%([^A-Za-z0-9]|$) is redundant, as %TOPIC%([^A-Za-z0-9]|$) already matches WebHome which is part of Main.WebHome.

Mh, the problem seems unresponsive to treatment See my discussion on BacklinksTemplate.

Added testcase, changed description and topic summary.

I fiddled with the suggestions given above and came up with the following expression in view.pattern.tmpl:

This expression fixes all testcases on BacklinksSearchTest except

• BackLinkReference6 which uses the $doPluralToSingular$ feature.
  • Fixing this seems difficult as the conversion algorithm has to be regarded.
• BackLinkNoReference as the search is case insensitive.
  • Fixing would invole setting the search to casesensitive=1 and changing %SPACEDTOPIC% (see TWikiVariables) so the initials are case included as both lower and uppercase letters.

The search string may also be useful for BacklinksTemplate.

BTW I'm not happy with the testcases flowing around in the Sandbox web as they are prone to change. I'd prefer to have them in seperate test web. And indeed, there is a ViewDEVELOP:TestCases web. How can non-elevated users propose test cases?

-- DanielKabs - 12 Jan 2006

backlink fix for 4.0.5

in backlinks.classic.tmpl, backlinks.pattern.tmpl, and backlinks.tmpl

- %TMPL:DEF{"searchallwebs"}%%SEARCH{ search="%WEB%.%TOPIC%([^A-Za-z0-9]|$)|%WEB%.%SPACEOUT{"%TOPIC%" separator=" *"}%([^A-Za-z0-9]|$)" type="regex" scope="text" nosearch="on" excludetopic="%TOPIC%" web="all,-%WEB%" }%%TMPL:END%
+ %TMPL:DEF{"searchallwebs"}%%SEARCH{ search="%TOPIC%([^A-Za-z0-9]|$)|%WEB%.%SPACEOUT{"%TOPIC%" separator=" *"}%([^A-Za-z0-9]|$)" type="regex" scope="text" nosearch="on" excludetopic="%TOPIC%" web="all,-%WEB%" }%%TMPL:END%

- $searchString = '\<'.$webString.'\.'.$topic.'\>';
+ $searchString = '\<'.$topic.'\>';

-- SteveStark - 28 Nov 2006

Example used above

BacklinksSearchDoesNotWork