I am now starting to use Google to search WikiLearn (and TWiki) because I find the built-in search facilities inadequate. There are some issues to address.

This is a "working page" -- once I understand some of the issues better, I may create RFEs for TWiki and move them to Codev or Support.

See also UsingGoogleToSearchWikiLearn.

See AboutThesePages.

Contents:

How to Use Google to Search WikiLearn

See UsingGoogleToSearchWikiLearn.

How Often Does Google Index TWiki?

I expect to do some testing if I can't find out any other way -- search for keywords for recent pages and see if they are indexed, and note the date (or days since creating the page).

Is there a way to get them indexed more often?

"Dynamic" Pages Should Not Be Indexed

I get too many results when searching twiki with Google. Some of them are not very useful at all, like the index of a "dynamic" page (like WebChanges), or any page with a search (maybe). I wonder (I don't know enough about HTML / the Web / TWiki) if I can put a no index (no robots, robots.txt, whatever it takes) on those specific pages. Probably not unless I create different templates -- I think that line has to be near the beginning of each web page.

No Cache?

Maybe I should specify "no cache" so my pages are not cached on Google. This might help address the problem of dynamic pages (described above), and might also alleviate other problems, like searches turning up old copies of pages which are no longer relevant (because in the current revision the content has been deleted, moved, or whatever).

There is work in progress to add to pages with url parameters so that SearchEngineIndexOnlyPlainView.

And you should also be using a robots.txt file to PreventGoogleToIndexRevisions and other non-content parts of the site (should work on other search engines provided their crawlers obey robots.txt).

Google Indexes All Domains -- Maybe Should Not?

Google indexes pages both under the twiki.org domain and the twiki.sourceforge.net domains -- not a terrible problem for me (especially if I specify the domain to search), but maybe Google would want to know that they are duplicates and not create two copies of the same index. (Well, except that I'd have a problem if they only indexed twiki.sourceforge.net and I specified twiki.org.)

Contributors:

• RandyKramer - 15 Nov 2001