Hi, everybody.

I am implementing a perl script to produce graphical clickable map of a Web (for an example see at http://twiki.dsi.uniroma1.it/twiki/view/Know/WebMap (sorry for the colors )).

I am using the wonderful graphviz package, that takes care of doing the graph layout.

Together with graphviz (see http://www.graphviz.org ) you can use webdot, cgi script that wraps graphviz and allows you to produce clickable graphs (and it caches the images!).

NOTE: use at least version 1.7.5 of Graphviz, earlier ones are without edge styles.

The idea can be extended to handle:

• a graphic map of web topics
• a graphic map of users/groups
• a graph of the "more related pages" to this page (see on the original Wiki http://www.c2.com)

My script produces a .dot file that lists all nodes, links and some layout information, webdot does the rest.

For the moment I am:

• clustering referred pages of each other web together
• coloring the node with the web color (thanks to ManpreetSingh for the color conversion code)
• highliting web's home pages
• use a template for the above points

I would like to:

• filter out unneeded pages

-- AndreaSterbini - 17 Jan 2001

There is some similarity to the capabilities and uses of TWikiDraw, I've just post a modification that allow clickable client side Image maps (TWikiDrawImageMaps). The Java software that makes up TWikiDraw has been used for graphs.

-- JohnTalintyre - 17 Jan 2001

Does TWikiDraw create the graph from the topics?

-- AndreaSterbini - 18 Jan 2001

It doesn't, it only provides a means of drawing the objects. The key advantage of the AT&T stuff would certainly be the ability to layout the graphs.

-- JohnTalintyre - 19 Jan 2001

Pseudocode (Actually perl)

sub maxof {
   my ($a, $b) = @_;

   return $a>$b?$a:$b;
}

sub minof {
   my ($a, $b) = @_;

   return $a<$b?$a:$b;
}

#  Converts rgb to hsv.  All numbers are within range 0 to 1
sub rgb2hsv {
    my ($r, $g ,$b) = @_;
    my $max = maxof($r, maxof($g, $b));
    my $min = minof($r, minof($g, $b));
    $v = $max;

    if ($max > 0.0) {
        $s = ($max - $min) / $max;
    } else {
        $s = 0;
    }
    if ($s > 0.0) {
        my ($rc, $gc, $bc, $diff);
            $diff = $max - $min;
        $rc = ($max - $r) / $diff;
        $gc = ($max - $g) / $diff;
        $bc = ($max - $b) / $diff;
        if ($r == $max) {
            $h = ($bc - $gc) / 6.0;
        } elsif ($g == $max) {
            $h = (2.0 + $rc - $bc) / 6.0;
        } else {
            $h = (4.0 + $gc - $rc) / 6.0;
        }
    } else {
       $h = 0.0;
    }
    if ($h < 0.0) {
       $h += 1.0;
    }
    return ($h, $s, $v);
}

-- ManpreetSingh - 19 Jan 2001

Andrea, this is great stuff! Especially if extended to show "more related pages". I can't wait to try it out!

I believe navigation will improve a lot if we can show a map of a web that shows relatives of the current topic in relation to the root topic - the WebHome topic.

-- PeterThoeny - 21 Jan 2001

There are plenty of very good navigation suggestions in Wiki and ModMod ... wonderful ideas ...

Should add:

• A Users Web (instead than the "Main" one)
• Filter out Users and Groups from the "Users" web from the graph (they produce a mess)
• collect useful routines in a nice module.
• add a "foundLinkHandler" to the API so that plugins can inform the rest of TWiki about links they produce.

-- AndreaSterbini - 21 Jan 2001

Have a look, I attach the script webmap its graph template webmap.tmpl and a page using it.

• MAINWEB topics are filtered
• first attempt to templatize graph specification

GroupsMap

I enclose also my very first attempt to a groupsmap script to produce a reasonable display of the groups relations (for an example see http://twiki.dsi.uniroma1.it/twiki/view/TWiki/GroupsMap )

• no template for the moment
• ellipses are groups, rectangles are users
• bold arrows for include relation between groups
• normal arrows for users included in groups
• dotted arrows for showing who has the right of changing the group definition (this way you can visually check if some user can upgrade his/her rights)
• now uses less clusters (nicer graph)
• now you can choose if you want to graph the groups only (groupsmap/groupsonly.dot) or if you want groups AND users (groupsmap/groups.dot)
• I have packaged the script + a sample web page ... just uncompress GroupsMap.zip in your twiki dir.

TODO:

• better visualization of the "upgrade my rights" check
• templatize it

-- AndreaSterbini - 22 Jan 2001

This is a very good feature indeed. I've tried a little bit with another script which produces smaller WebMaps. It shows all pages you could reach within two clicks from the current page.

• the script step is based on the view script, but does read another template step.tmpl
• the script stepmap is based on webmap, but uses also the topic name to find the starting point for the graph. It now uses a stepmap.tmpl
• add an Step entry to the bottom line in the view.tmpl to get to the step URL:
  

| <A href="%SCRIPTURLPATH%/step%SCRIPTSUFFIX%/%WEB%/%TOPIC%">Step</A>

• copy the step.tmpl and stepmap.tmpl into the template directory.
• now you can click Step on every page to see the nearest pages

The script isn't perfect, but perhaps you can imagine what is possible with this graphviz tool within TWiki.

TODO:

• there should be a library function to catch all WikiTopics from a text file. The grep in the webmap (and also) stepmap script doesn't work for all links yet.

-- StefanScherer - 26 Jan 2001

Yes, probably we should render the topic and then collect all html links ...

WebMap TODO: filter also some topics (WebPreferences, WebStatistics ...)

GroupsMap TODO: show also a node for each web, showing the default web permissions (i.e. the ALLOWWEBCHANGE preference)

Stefan, I suggest you change the code to use a stepmap.tmpl template ...

-- AndreaSterbini - 31 Jan 2001

Yes, you're right. I've changed it already. I now use getRenderedVersion and after that the hyperlinks to the view script will be translated into a graph. So the stepmap also works on pages with an %SEARCH% inside.

-- StefanScherer - 01 Feb 2001

Since http://www.graphviz.org not always seems to work, you could also try http://www.research.att.com/sw/tools/graphviz

-- HansDonner - 15 Sep 2001

Is this a plug-in?

-- MartinCleaver - 22 Oct 2001

The feature I wanted to create is based on GraphViz, is a WebDOTPlugin.

-- WayneScott - 14 Nov 2001

I was just looking at http://www.memes.net/ mentioned in LucidFriedEggs web site. They use TouchGraph http://touchgraph.sourceforge.net/ which is a free java based graph layout tool. Think of it as a realtime graphviz. It has some really nice features incuding:

• the graph is re-centered around a selected node
• only nodes 1, 2, 3, ... (user selectable) hops away from the current center node can be shown. Kind of a local picture of the graph tree that can be opened up to larger views.
• popup text when you hover over a node. This allows document summary to be displayed.
• the ability to color edges and nodes based on criteria.

It reminds me of a similar commercial tool Inxight Star Tree http://www.startree.inxight.com.

-- JohnRouillard - 24 Nov 2001

I just attached my updated version of WebMap (as WebMapJC.zip) that works with the newer versions of TWiki.

-- JohnCavanaugh - 26 Feb 2002

John: WebMap sounds like an add-on that should go into the Plugins web as Support.WebMapAddOn.

Related to WebMap, AlexShapiro's TouchGraph can visualize MeatBall's wiki's pages. See MeatBall:TouchGraphWikiBrowser.

-- PeterThoeny - 15 Mar 2002

Peter: WebMap in many ways needs a complete overhaul. Granted it sorta kinda works now but things need significant cleaning up, therefore I wouldnt call it a plugin more of a demonstration.

Things that should be done.

1. Remove dependence on WebDot (it requires people to have TCL) by using pure perl & the GraphViz perl library.
2. Create a generalized mechanism to determine what pages refer to others. Perhaps store in a csv or SQLLite database. (Probably would require an OnSave hook)
3. Create a page to host the webmap that has controls to allow you to alter the presentation. Im thinking about things like "Show/Dont-Show Pages in other Webs", "Show/Dont-Show Links to Users", "Show/Dont-Show Topic Modified in last xx Days", "Color code pages by WebStatistics" etc.

Longer term I would like to have a panel on the side that showed "1 degree of separation" from current page. I described it in HowToShowParentTopics.

-- JohnCavanaugh - 15 Mar 2002

I think this is a great project and attempted to check out the working example of WebMap but the url links do not seem to be working.

-- LynnwoodBrown - 16 Mar 2002

See Also http://www.usemod.com/cgi-bin/mb.pl?TouchGraphWikiBrowser for a very cool implementation of a node browser for MeatBallWiki. It's based on JGraph (methinks). It's a little standalone app that reads the MeatBallWiki database and generates a very cool NodeMap of the data. Download and play!

• See the local page on TouchGraph first. -- MattWilkie - 19 Jun 2002

-- DennisDaniels - 22 Mar 2002

I tried using webdot but I just couldn't get it working... and I noticed a Perl GraphViz module, so I started hacking. Anyway, what I ended up with was a two-level SVG topic map that was clickable to switch topics. It's been useful for me because I can see what pages linked to the current topic as well as see topics that are just out of sight (two topics away). So I uploaded what I created, not much, not nearly a "finished" piece of work, but maybe useful to somebody.

• whereami-0.01.zip: Perl/GraphVis implementation

-- RobertHanson - 17 Jul 2002

Robert, could you attach a screenshot or list a url for what you did?? It would make it easier than installing it just to take a peak.

-- JohnCavanaugh - 17 Jul 2002

Here is a screenshot of the SVG file, it's internal only and not available on the net. This one in particular is embedded on my "JavaLanguage" topic page. The current topic is in red surrounded by linked documents (link direction not specified, up to two topics away). The best part about SVG is that you can zoom in or out without losing clarity, and the Adobe plugin also allows you to pan. The script could easily be changed to output a PNG or other format by changing one line... but then it wouldn't be clickable (without some work), and just didn't look as nice.

• Example of "whereami" SVG output:
  

-- RobertHanson - 17 Jul 2002

Interesting. One of the things I always thought would be cool is some type of child window that only shows this 2 degree of separation diagram and automatically is updated refreshed as you move around the the main browser window clicking on links. I guess this is sort of a dynamic graphical birds eye view of the wiki site topology.

Im not sure how this could be integrated automatically into twiki. Im guessing it would definitely require some javascript to be built into one of the skins to keep autorefreshing the child window.

The other thing is Im not sure how useful this would be in real world situations. But I do know that it would demo very well and demos often help gain support/sponsorship from the upper level managers.

-- JohnCavanaugh - 17 Jul 2002

Something like the auto-updating map should be fairly easy to implement, all you would need to do is add a single line of JavaScript to the <body> "OnLoad"... something like OnLoad="JavaScript: adframe.location=('map.svg?topic=%TOPIC...')" (or maybe a few lines).

But like you mentioned in an earlier post (from March), some mechanism would be needed to store relationships. Right now I am generating them on the fly, but with an always present map it would be even more important to store the links so that they don't need to be regenerated each hit.

But like you said, I don't know how useful it would really be. It may even be considered annoying to some users... especially if they are on a dail-up.

-- RobertHanson - 17 Jul 2002

See WhereamiWindows for a IIS and ActiveState usable Version ;-). Additionally implemented some caching features.

-- MarkusKling - 21 Jul 2002

>>The other thing is Im not sure how useful this would be in real world situations.
This is exactly the navigation which PersonalBrain uses. I found it very useful, as long as it is fast and integrated. It is not with http://www.thebrain.com on the web. And I do not even dare to think, how it would perform in TWiki, given how slow it is with just text...

Tried TouchGraphTopicList - this really really helps!

-- PeterKlausner - 22/23 Jul 2002

Dear all,

I know that this topic is rather old and that we have some tools already available but this app -> http://hypergraph.sourceforge.net/ seems to be worth the look. It looks like The Brain mentionned above. There is an example with a wiki, will Twiki be capabe of that ?

-- EricCharikane - 03 Oct 2007