Other Data Types

Wikis like TWiki are great for manipulating text, but not so good for other data, such as

• drawings
• spreadsheets
• PERT charts, etc., from tools like Microsoft Project
• basically, anything that there is an external application for

We can always attach such documents.

• Minor cost: we don't see the attached document inline * except for things like TWikiDraw, which attach both a GIF that can be viewed inline and the source code * this is only minor because we can always click on the attachment link; most browsers will then display the file, if it is a datatype known to your system. Sometimes in the browser, sometimes by sabing to a temp.
• Major cost: editing the attachment is a pain

TWikiDraw is nice, but VISIO is better...

For one particular activity, drawing, we have the TWikiDrawPlugin (and TWikiDrawSvgPlugin).

Which is great, but

• Fancy drawing programs like VISIO are better
• Even Microsoft paint is easier for quickly recording pictures
• Lately, I have been using pen drawing tools such as
  • Microsoft OneNote
  • Microsoft Journal
  • Corel Grafigo

Also, I frequently find that I want to edit the drawings without being connected to a webserver or wikiserver, i.e. standalone.

Bottom line: a built in tool is a great lowest common denominator. But it would be nice to be able to use fancier toolswhen available.

PUT to the web

I think that the HTTP PUT method is intended for use in such circumstances. Basically, the web is a filesystem: GET reads the file, PUT writes the (entire) file. (POST is for forms.)

Most applications do not support HTTP PUT, although more and more do. When they do, great.

TWiki should be extended so that a PUT to an URL corresponding to an existing attachment replaces the attachment. Probably, also, so that a PUT to an URL can create a new attachment.

Getting Applications into PUT

So, I'd like to use an external application, one that isn't necessarily aware of HTTP PUT, to edit attachments.

Two methods:

Links and/or Macros

The document data itself can contain a link or macro that causes the PUT.

Possibly by invoking an external command that does the PUT (e.g. wput, imstead of wget).

The document may need to know the publish URL.

PUT wrapper

Instead of calling the native application (Word, FrameMaker) to display an attachment, call a wrapper that will just invoke the native application... but which, if invoked on a temporary copy of an attachment, knows to use PUT after a save.

Gross Hackery

I am suggesting gross hackery. The right thing to do is have the web appear as a filesystem via HTTP PUT and GET. But stuff like the above may be hackable for particular apps - e.g. anything supporting VBA. Easy on UNIX.

The Ultimate Right Thing

The ultimate right thing is

• wiki should support arbitrary attachments
• for important data types (text, drawing) wiki should have a built-in display and edit capability
• it should always be possible to override the wiki built in tool to use a fancier external tool; both for display and edit
• the external toool and the wiki built in tool should be able to exchange editable data files.
• it would be nice to be able to script each external tool into producing both the fancy format and a format that wiki knows how to display (e.g. .GIF)

-- AndyGlew - 23 Sep 2003

Commentary

Y, I concur. I mentioned this in less detail at WebDAV

-- MartinCleaver - 24 Sep 2003

Oh yes! That would be very useful. And if understand correctly, the "hack" could equally apply to the topic text itself. This would resolve DownUploadForEditing

-- PeterKlausner - 25 Sep 2003

I've learned more than I wanted to about HTTP GET, PUT, and POST while trying to RedirectWikiUrls.

-- AndyGlew - 27 Sep 2003

New Thoughts on Server Side File Conversions instead of Client Side

Haven't made any progress wrt ThoughtsAboutUsingExternalApplications for years, and then started getting new ideas.

All of my earlier thinking on this topic has been client oriented: I was trying to figure out how to get the client to generate something acceptable to TWiki. E.g. I wrote Visio macros (VBA) to automatically save a Visio drawing both as a Visio .VSD file and a .GIF, and then I tried to upload both to the wiki. Now, I was able to semi-aitomate the double upload, and I got the VBA, but I was never able to get the twain to meet: I was never able to make the process of uploading two different attachments a 1-button thing. (Web-browser security gets in the way, quite legitimately.) And even if I had, this was not a universal solution: it would be hard to persuade other Visio users to run my VBA macros. (Again, a legitimate security concern.)

This is where I was stuck for years.

Server Side Conversion Plugin

But now I realize: we could do the conversion on the server. Via a plugin.

E.g. I could (should) write a plugin that uploads a Visio .VSD file, and then processes it locally, on the wikiserver, to generate an accompanying .GIF. Attach both to the page. The .GIF can be seen inline; the Visio .VSD is also attached, so that vector graphics editing can be done.

It still would not be entirely 1-click. Sure, you can click on the Visio attachment, and have Visio open it on your client. But when you save, I believe most tools like Visio save to the local disk, e.g. to the temporary file that was created by the browser for the external application to run on. You would still have to save the file, and then ask wiki to upload it.

But: only upload the external application file format. E.g. the Visio .VSD drawing, the Word .DOC document, the PowerPoint .PPT slideshow, the Excel .XLS spreadsheet, the FrameMaker .FM or .DOC, ...

Have the wikiserver (via this hypothetical plugin) recognizze the filetype, and then convert it, if desired, to something more wiki friendly.

E.g. for my canonical example, recognize the Visio .VSD drawing. Attach the .VSD to the wiki page. Now, convert the .VSD to a GIF, on the wiki server, and attach that.

Q: how do you convert a .VSD to a .GIF on the wiki server?

(0) if the wikiserver is running on windows (does TWiki run on Windows) use the native Windows app, if you can script it.

(1) if the twikiserver runs on UNIX, and if it is a file type that OpenOffice supports, use OpenOffice. (Hmm.... I know that OpenOffice handles older Word, PowerPoint, and Excell files. I don't know if OpenOffice handles Visio. Anyway, the idea is the same.)

(2) if the twikiserver is running on an OS or microprocessor where the external application does not run, try to find a web service for it.

E.g. the web service server might run on a Windows box. The UNIX based twikiserver might send the file to the web service server, which would do the conversion, and return the .GIF.

Now, I do not know of a web service server - calll it a conversion server - that does this conversion, Visio to GIF. But I can see how to write it. Moreover, I am aware of several web service servers that convert Word and PowerPoint to PDF, which is useful in itself.

So, the wikiserver plugin could accomplish the conversion of the foreign data file format by any of these means.

• Maybe it would convert a Visio drawing to a .GIF, and inline the .GIF on the wikipage.
• Maybe it would convert a set of PowerPoint slides to a GIF. (Are multipage GIFs or TIFs viweable inline on web pages?)
• Heck, maybe it could take a Word document, use Word or OpenOffice to convert it to HTML, and then process that HTML to put on the wiki page (either as HTML, or maybe by extracting the text). Maybe this would be a way of allowing Word or some other editor to edit Wiki pages.
• Maybe it would make a more ubiquitous file format such as (gag) PDF available.

Wiki User Interface Special Handling in Plugin

The plugin would want to handle such "related file attachments" specially:

• if you upload the original file format, say a .VSD, it would convert anew and overwrite the old .GIF
• if you try to upload the .GIF, it would warn you
  • typically, delete the old .VSD, since it is stale wrt the .GIF

More special handling for foreign file types that have been converted to wiki text or HTML via this conversion process

• if you try to edit the wiki text warn
  • optionally switch to editing the original, e.g. Word
  • optionally remove the stale Word
• if you edit the Word, optionally overwrite the wiki/html

ISO save direct from app to wiki

Server side file format conversion would remove a lot of the annoying work from the client. But it still falls short: the user is still not able to click to open the foreign file in its native application, edit, and then save inside the application and have it naturally appear on the wiki.

(This sort of save direct from app is supposed to work in SharePoint. But it doeesn't seem to be working on my team's SharePoint site today.)

Problem: typically such files are placed into a temporary file by the browser, and then the app is invoked on that file. Usually the app cannot save to that temporary file, but even if it could it does not automatically happen that the browser picks up the saved temporary file, and puts it back onto the server.

Some sort of web based filesystem - WebDAV? - might allow the app save to take effect. The app would open the file from the WebFS, and save to that. Whe wikiserver would be intercept the save, and post as an attachment. That's where we were years ago. Not too many changes to apps, if it can acces a WebFS. But arranging such access may be problematic.

Changing the apps to support HTTP PUT might work - but every app would have to change.

Possibly we could get away with modifying "just" the browsers. If the browser recognizes that the temporary file it gave to the app has changed, it could try to execute an HTTP PUT back. In theory, it might be easier to change 1 browser than N apps; but, there may wellbe more broswers than apps.

I do not yet have a direct-save-from-appt-to-wiki solution. But I keep thinking about it. And server side file conversions, possibly using a web conversion service, could be part of the story.

-- AndyGlew - 15 Jul 2005

From what I have read on this topic, you like live in a homogenous Microsoft Software world. This makes suggesting Server side conversions much easier, as there are fewer applications of merit >grin< and thus file formats that you need to support.

even so - sounds very interesting, and would certainly be a killer app!

when I edit a viso/excel/word doc thats on my companies sharepoint server, saving sends it back to the server - so there must be a mechanism that you can hijack

-- SvenDowideit - 16 Jul 2005

minor semantical change to sven's comment; Sven, delete this comment after you read/verify it. [matt]

An ambitious project Andy. Go for it!

-- MattWilkie - 17 Jul 2005

Andy - I think you and I are talking about the same thing. See AttachmentRenderings

-- MartinCleaver - 18 Jul 2005