Submit Topic by Email

• Create a new topic or add text to an existing topic by sending an email to twiki@yourdomainPLEASENOSPAM.com .

-- PeterThoeny - 06 Sep 1999

---

Possible Applications:

• Alternate interface for topic submission
  • Offline modification
  • Transfer from e-mail (creating a topic from an incoming customer request)
  • Transfer from other system (via e-mail notification)
  • Faster Attach (possibly a work-around for browser issues with very large files)
• Convenience method
  • More familar to auxilary users (sales submits the initial project proposal)
  • Add topic information, updates, or suggestions while reading/writing e-mail
  • Round-trip system
    • Download topics remotely (for edit)
    • Review comments or complete topic editting
• Mailing List Archive
  • Collect e-mail statistics
  • Combine with notification by e-mail to make TWiki Listserve
• Issue Tracking
  • Create a table of submissions
  • Create a new issue topic for each submission
  • Issues like Resumes submitted, Customer RFQs, Information requests, Action Items
• E-mail based requests
  • Automatic responses maintained as wiki topics
    • First line customer support
    • Search by e-mail
    • Download topics by e-mail
  • Control by e-mail
    • Issue tracking state updates
    • Approvals by e-mail

Plugin Functions:

• Get incoming messages (MailToTWikiAddOn, MailInContrib)
• Authentication to determine Wiki User (probably the same above)
• Filter messages for action (Use TML as a script?, something like NamespaceManagerAddOn)
• Subst and Extract (FilterPlugin?, TopicCreatePlugin)
• Notification for response

-- JadeCravy - 08 Jul 2006

---

I am struck by a sudden vision of a Wiki mailing list archive. Many of the components are already in place -- notification on changes (each list would have a web, each email subject that's not re: [topic] it's own topic), and if you add this, changes of notification (email). It should also be possible for the archiver to insert email responses contextually, if straight chronological ordering is not desirable. The idea being that a lot of mailing list traffic (well, for JOS, anyway) is discussion of a particular subject, attempts to reach a decision on a design, etc, things that would greatly benefit from tighter integration with the Wiki. I am no sure, however, how well this fits with the WikiWay, which may suggest that such collaboration ought to be done a Wiki. An (implicit?) APPEND tag would be useful for this, as well as normal email submissions. (That is, add the following to the end of the page, rather than replacing the whole thing.)

-- mailto:jquinn@usersPLEASENOSPAM.sourceforge.net - 12 May 2000

---

A Wiki mailing list archive has already been implemented by WardCunningham for archiving the WikiForum list. The archive is at http://c2.com/w2/wiki/RecentChanges . Features and spec AFAIK:

• Create one topic per thread
• Additional messages to a thread are appended to the same topic.
• How to determine a topic name (e.g. a thread):
  • Use the email subject of a message and
  • Strip commmon prefixes like 'Re:', 'Fwd:',
  • Delete all non alphanumeric chars like '[', ':'.
  • Create a WikiName from the subject by
    • Changing the first char of each word to upper case,
    • Changing all remaining chars to lower case in case word is not in WikiNotation,
    • Deleting spaces.
  • Example email subject:
    • Re: [WikiForum] Progress report on "WikiServlet"
  • Example topic name:
    • WikiForumProgressReportOnWikiServlet

The resulting archive is OK, but it is not as organized as a Wiki site. This is because:

• Cross linking is not done as frequently (though it could done by using WikiNames in the email)
• A new topic is sometimes created for the same thread. This happens in case a reply contains unexpected strings (like 'AW:' instead of 'Re' for German email clients)

TWiki could be extended to have the same functionality. I would not recommend to allow email submissions to any TWiki web, this is too chaotic. There could be webs with a special flag to allow email submissions. One practical application would be to have a Devlist web that archives the TWiki-dev list ( twiki-dev@listsPLEASENOSPAM.sourceforge.net ).

-- PeterThoeny - 16 May 2000

In terms of knowledge management email (and news) archives are good. Deja is an example of that. What would be nice is for the information in email threads to be leveraged into more stuctured constructions like TWiki. There is certainly room for discussion of ideals in that direction, rather than the straight archiving which is done already quite well.

It might not be much more that pulling email threads into seperate Webs allowing them to be edited on a backend before being presented to the public site. I imagine that TWiki is already used in a manner like this, but with more manual labour.

I guess the questions is what workflow tools would make it easier to do this. Things like automatic keyword WikiWikiName indexing.

-- NicholasLee - 21 May 2000

I am going to use TWiki for making a collaborative web for teaching at my University, so I am very interested in having this enhancement implemented.

Is there some code already available? Should I adapt the WardCunningham's code?

-- AndreaSterbini - 04 Sep 2000

AFAIK no code for TWiki is available. You could ask WardCunningham if he is willing to share his code and go from there.

-- PeterThoeny - 06 Sep 2000

Quite an interesting problem is that of how to incorporate mail and its replies. We would want the replies to follow the original mails, perhaps to be tacked on to the original.

Perhaps we can use a threaded mailing engine to group the emails?

(I had a feeling that there was another topic on this subject but I can't find it.)

-- MartinCleaver - 11 Jul 2001

Another idea: we could keep a topic that mapped mailed-in Subject names to the topic name to which those emails should be appended.

-- MartinCleaver - 16 Aug 2001

It4s amazing: this is one of the first ever topics on TWiki.org!

I4d like the group to dwell for a moment in how much of your entire organisations business is held together using email and how much of that email is privvy to just the recipients. Unlocking that email, allowing it to be cross-referenced and searchable by all and any would in itself present massive benefits to most organisations. Wow.

-- MartinCleaver (in enthusiastic mode) - 23 Aug 2001

A variant of appending by emailing: AddToTopicLikeBackflip

-- MartinCleaver - 07 Sep 2001

Okay guys, I've got some development effort over next week to have a bash at implementing this as a PlugIn. Can someone tell me what API should be used to append a new revision of a file?

Func.pm does not have anything obvious.

Thanks.

-- MartinCleaver - 04 Oct 2001

Okay, we are currently running a project to implement this. Can anyone offer a business case that I can use at my company to persuade them to release the improvements we've made? Thanks.

-- MartinCleaver - 29 Oct 2001

While using open-source software in a big company the argument that I always had the greatest success with was to reduce maintenance in the long run (essentially sell the open-source paradigm). Whatever business advantage you may get by holding on to changes is far outweighed by the expense of maintaining/adapting the features. In my case I worked for an avionics company and was occasionally modifying compilers/libraires, but my job was writing command and control software. Unless you are in the compiler (or TWiki) business the cost associated with long-term maintainence of the changes (and use of non-standard installations) provides no real business advantage. I.e. they made money on command and control systems, not compilers.

-- DaveAlsup - 02 Nov 2001

Thanks, I'm aiming to put the case forward by the end of the week.

-- MartinCleaver - 19 Nov 2001

As it turns out, we've got a need for this exact feature, both for mailing lists, and to do general topic submissions via email. We're finishing up our MultipleAttachmentsAtOnce patch, which makes it SO much nicer to upload a whole bunch of attachments at once.

MartinCleaver, what's the status of having this code become generally available? We'll be donating our patch to the main codebase in the hopes that it becomes generally integrated, and we don't have to spend an hour or two with every new TWiki release to keep it up to date. It's all about the reduction of effort.

Thanks, -- VitoMiliano - 13 Jan 2002

Aghh. So annoying. Okay, Vito, status is as follows:

1. I no longer work at Andersen.
2. I am currently travelling, I'm a Brit but am in New Zealand at the moment. I expect to be travelling in total for up to six months.
3. Andersen gave me permission to submit the mods we made but on the understanding that I am the sole contact for those changes and that they are not associated with them.
4. I have packaged and released some of the changes and have a few more to do.
5. This evening I got hold of the code for our JavaPaste utility (see that topic)
6. I don't have the SubmitTopicByEmail code yet but I expect to within the next few days.

The annoying thing is that I think we may have duplicated some of your efforts from MultipleAttachmentsAtOnce in our JavaPaste utility.

I am sorry that I didn't inform the community that we were working on JavaPaste, I was already conscious that I had promised a lot and didn't want to further promise without showing that I was able to deliver on my promises.

-- MartinCleaver - 14 Jan 2002

JavaPaste sounds like it requires Java. If that's the case, don't worry, we wouldn't have used it anyway. ;-D

Great that you'll be posting the SubmitTopicByEmail code! Looking forward to seeing it. I just put up our MultipleAttachmentsAtOnce stuff, so maybe the TWiki gods will see fit to include it in the next release. fingers crossed

Is TWiki still in use at Andersen?

-- VitoMiliano - 14 Jan 2002

Your quite right JavaPaste does require Java. It is worth it though :^)

See MailInAddOn for the code...

TWiki is still in use at Andersen, to what extent it will be used only time will tell as I was the main advocate. We had 80+ people registered with 10 or so enthusiasts so I'd say if they like it the masses like it and Andersen continue to not provide anything better, Twiki will spread. That said the firm is very good at rationalising the software that they keep.

-- MartinCleaver - 18 Jan 2002

One thing that I've found quite important is to actually train people in TWiki, either one-to-one or in groups of five or so, with a lot of TWiki practice in the course. This has greatly raised awareness and usage of TWiki, so it's the single thing I'd recommend to improve usage. See HowToGetInternalBuyInForTWiki for more on this topic.

The submit-by-email feature sounds very useful, so people can send in updates via email even if they are on an offline laptop - Outlook and similar email tools have strong offline syncing, so it's good to piggyback on this until ReadWriteOfflineWiki is possible.

-- RichardDonkin - 19 Jan 2002

Eek! UDDI and WebServices and SOAP, oh my! Isn't this a bit much just to be able to send an email to TWiki? I feel kinda silly doing all this extra stuff.

Although I do understand the point of them, I just have no need for any of it. Perhaps I'll hack up the plugin to function standalone; otherwise I'm going to feel obligated to make use of all the extra WebServices functionality.

Thanks again, Martin!

-- VitoMiliano - 19 Jan 2002

Don't worry, its all very simple. It just looks like an API to the end programmer.

That said, you should be able to use the MailInAddOn without the WebServicesAddOn, no hacking required. Just call MailIn::send(SmtpMailBody) and MailIn::retry(). Let me know how you get on!

-- MartinCleaver - 20 Jan 2002

For what it's worth, I've been playing with adding SubmitTopicByBloggerApi to UseModWiki. I imagine it would work for TWiki, too. Much of this has come from me playing lately with RadioUserland and MovableType for my own projects. Been seeing that a sort of convergance between weblogs, wikis, web services, RSS, etc.

-- LesOrchard - 20 Jan 2002

I would like to try this out too. (I am not a regular here, but we are trying to use TWiki for CMM Level 3 standards here - apart from other non-serious use.)

Meanwhile, I was thinking about another approach: Access IMAP mailbox via a plug-in in such a way that sets of related messages appear as a single topic. Actually I will refer to these topics as 'Virtual Topics' because you can pull data off any store (database, IMAP, LDAP etc.). Of course, in the read-only mode, it should work just fine.

One of the main use of this capability would be to use TWiki to organize emails into related areas within a topic. We will also be able to refer to TWiki topics from within emails. I will elaborate these points if there is some interest.

-- VinodKulkarni - 08 Feb 2002

Your IMAP ideas do look interesting - presumably a virtual topic has the same content as a current ThreadMode TWiki page, but can be accessed as an IMAP folder as well?

Also, I'm interested in how TWiki would fit into a CMM Level 3 culture, which presumably is very process and quality centric - we have a quality and process initiative in my company but TWiki raises concerns with the Quality people about possible uncontrolled creation and duplication of information. Perhaps we could spin this off into a TWikiAndProcesses topic?

-- RichardDonkin - 08 Feb 2002

I have such - whatever this means now - a thing working for my private weblog. You can find the perl code

here

I still need some way of having others inserting comments separated from my weblog. Something like NotePlugin

-- FrankHartmann - 24 Apr 2002

One thing I neglected to say is that the MailInAddOn took about 3 weeks of coding. Its quite solid, provides features such as presenting queued messages as topics in a separate web.

I guess I'm really saying that I'd like people to play with it

-- MartinCleaver - 07 May 2002

Gosh, I'm sorry. I only just realised that I never posted the testing files. More soon. Watch this space.

-- MartinCleaver - 11 May 2002

I have the feeling that a complicated part of the SubmitTopicByEmail will be the connection of the logbuch script or MailInAddOn to the mail transfer agent. I saw that in MailInAddOn Lotus Notes and a unnamed POP3 Server are mentioned, which enable this approach, but no recipies or examples how to set this up are given. In logbuch script I described the setup for the Exim MTA, which is the standard for Debian based systems. If more of these possibilities exists I would like to see them on a dedicated topic. What do you think?

-- FrankHartmann - 13 May 2002

Seriously, this should be really simple. Just a procmail recipie.

I would guess that this would work. Save as .procmailrc in a user account called twiki

:0:
| /var/www/twiki/lib/TWiki/Plugins/MailInAddOn.pm (or whatever it is called)

This recipie would put the mail onto stdin. I think the script takes it off okay.

Then mail to twiki@yourdomainPLEASENOSPAM.com Subject dictates the topic. Needs to be fully qualified.

-- MartinCleaver - 14 May 2002

I've just written a very dumb pop2twiki MailIn program. See MailInAddOn.

-- MartinCleaver - 19 May 2002

Martin and Frank, your implementations look very nice. I'll have to more closely review the MailInAddOn to fully understand it's features. There seem to be many. The KISS approach by Martin is a useful reference as well.

The feature you have both implemented has been discussed for quite awhile. It is an intriguing interface between TWiki web and email communications. Why do you two feel it hasn't been used much?

-- GrantBow - 02 Feb 2003