Tags:
notification1Add my vote for this tag create new tag
, view all tags

Consolidate Notification

Applying the ExplorationThenConsolidation paradigm to MailNotificationEnhanced, YetAnotherNotifySystem, NotificationPlugin and ImmediateNotifyPlugin, we have now explored the possible space for notification and come up with some important features.

By consolidating we avoid confusing administrators as to what they should install and lessen the risk that the administrator will need to change his or mind consequently creating rework or confusing end users.

Note to editor: Check EmailNotificationEnhancements, NewEmailNotificationSystem ,

Current Status and Future Plans?

I invite the proper authors for this topic to replace this entire section with the requested status after having digested the text in this section.

I'm a potential, new TWiki admin trying to determine which email-notification mechanism to use. From what I can tell, MailerContrib would be recommended (although I may try an immediate-notify flavor because of my requirements).

Is the result of this effort targeted for Dakar or another release? Do we project it would be a separate plugin that I could interchange with something (like MailerContrib) with minimal impact to my system?

For what it's worth, "we don't know anything yet" is an acceptable answer.

Related topics: StatusOfDakarRelease CurrentStateOfEmailConsolidateNotification

-- MattEngland - 08 Apr 2005

Overview of existing solutions

The variations I have seen are MailerContrib, PerTopicWebNotifyReports, YetAnotherNotifySystem, ImmediateNotifyPlugin, ImmediateWebNotifyPerTopic, MailNotificationEnhanced. These are also listed in the related topics field below.

To give a high-level summary of similarities and differences...

  • MailerContrib:
    • Generates the notify topic as a search and thus can handle notifications deriving from multiple sources (such as action trackers or queries).
    • Allows descendents of topic to be includes, and allows topics to be excluded.
    • All subscribers are subscribed on the same schedule - i.e. it is not possible to update some daily and others weekly.
    • Syntax is a natural extension of the existing syntax.
    • Replaces existing mailer (or can coexist).
  • PerTopicWebNotifyReports, by Main.ShawnBradford:
    • Available for Dec 2001 release
    • Adds a {$topic}WebNotify alongside every topic
    • Adds syntax to designate individual topics and exclude topics from a per-web notification.
  • PerTopicWebNotifyReports, by Main.JosephWasson:
    • Available for Dec 2001 release
    • Extends syntax in the WebNotify to designate individual topics and exclude topics from a per-web notification
  • PerTopicNotificationAddon (ThomasWeigert)
  • ImmediateNotifyPlugin: First (29 Dec 2000) implemented by Sven as a script in ImmediateWebNotifyPerTopic, this was re-implemented by WalterMundt as a plugin.
    • Tested on 01 Feb 2003 release
    • Subscriptions go on WebImmediateNotify topic
    • Plugin supports immediate notification of changes to topics in web (by adding an after-topic-save handler) and via a patch on ImmediateNotifyPluginDev
    • Does not support notification on attachment upload
    • Duplicate of code found in mailnotify.
    • Has a modular architecture of its own: "plug in" of new delivery methods by just writing the glue code between the notifier library and the notifier plugin (at this point supports SMTP and JABBER)
  • MailNotificationEnhanced:
    • There is indication that this is working with Cairo, but users have reported problems.
    • Consistent with existing mailer.
    • Provides a general exclusion list of topics that should not cause notification (e.g., WebHome) on a per-web basis.
    • Allow topics to be designated to be included or excluded from notification by matching against topic names, topic content, or topic author, using regular expressions.
    • Notification templates can be chosen an a per-user basis.
    • Keeps copy of sent mail, if desired.
    • Can look up user email address in LDAP server.
  • YetAnotherNotifySystem:
    • Has been updated to Cairo.
    • Replaces existing mailer.
    • A code fork of MailNotificationEnhanced, focus was on simplifying the syntax of the WebNotify topic. Compared with MailNotificationEnhanced, this system adds:
      • a simpler syntax for per-topic notif settings
      • a completely refactored code for notif settings API
      • a new "Subscribe" button on all topics of the Wiki
      • backward compatibilty with current TWiki system
    • It removed:
      • LDAP support (candidate to be a TWiki-wide infrastructure component)
    • Provides simplified syntax consistent with existing and supports per-topic notification, topic names with wild-cards, and notification on a complete topic tree. "The user settings syntax is much simpler than MailNotificationEnhanced mainly due to the removal of features not useful to my users: per user mail templates and complex topic patterns."
    • Allows to exclude topic changes by the users.
    • Allows notification on all new topics.
    • Adds a "subscribe" link to the tool bar of the view template.
  • ImmediateWebNotifyPerTopic:
    • Patches to be added to the save method to cause notification upon topic change immediately.
    • Users to be notified are indicated via tags in the topic.

-- ThomasWeigert, MartinCleaver - 10 Oct 2004, 6 Jan 2005, 7 Jan 2005

(Please intersperse into this list)

Feature table

Each of these has something to offer.

Feature Plugins.MailerContrib Codev.PerTopicWebNotifyReports Codev.MailNotificationEnhanced Codev.PerTopicNotificationAddon Plugins.ImmediateNotifyPlugin Codev.YetAnotherNotifySystem Codev.ImmediateWebNotifyPerTopic
TWiki version supported              
Compatibility with existing webnotify system DONE ALERT!          
Designation of individual topics              
Subscribe interface in view topic              
Exclusion of certain topics such as the Web* topics ALERT! CC1            
Notify on topic descendents changing DONE            
Notification of all new topics only NEW   DONE        
Notification of the next change of this topic only (or until I view it again)              
Exclusion of topics changed by user (NOTMINE)              
Single notification system for action trackers, nofify, project tracking, etc.              
Independent schedules for different people NEW            
Support for AfterAttachmentSaveHandler              
Send the actual changes, including any updated attachments when it notifies.              

  • CC1 - I see the value of the "don't subscribe me option". Perhaps someone will be prepared to fund me to implement it (gotta eat) -- CrawfordCurrie - 05 Jan 2005

Notes:

Process

  • Someone needs to own this. Any volunteers?
  • Crawford has done a lot of work on this but is overloaded with other TWiki responsibilities. 1) Its only fair someone else takes this on. 2) It will likely get done quicker 3) the person doing so will have more influence over the result.
  • We need someone to 1) continue to integrate the comments in these topics into a coherent functional requirements document 2) produce a gap analysis WRT what's implemented 3) priortise easy wins 4) allocate work
  • Any pledges of funds (in, say, US$40 chunks) is likely to help this happen

Targets

Who Amount wanted What functionality you would be prepared to work on for this amount

Pledges

Who Amount pledged Functionality you'd like to see

Overview

A means to subscribe to notifications on topics.

Current situation

In many firms not everyone is interested in all the topics in the web and so are reluctant to subscribe to WebNotify. Those that do risk being bombarded with irrelevant updates causing disengagement from the content on the wiki.

Organic growth has led to many implementations being coded. These competing implementations now fragments effort and confuses users. The dissipated effort leads to time wasted trying to reconcile solutions.

There are by now three (corrected: six) variations on the theme of detailed mail notifications that have been submitted, in addition to the core code. It is very confusing for a newcomer to decide which one to adopt. Clearly, per topic notification and other control over notification at the user level is extremely useful. (e.g., in our installation, we have a link at the bottom of each topic that allows the user to add this topic to the mail notify topic automagically.) I'd suggest we pick one of the variations proposed and move it into core. I'd surmise that just about everybody has installed one or the other of these addons at their system....

-- ThomasWeigert - 09 Oct 2004

Here is the plan: This contrib is a logical extension of the core notify logic, so it will be replaced with this code. One additional Plugin should provide the UI magic that makes the subscription easier.

-- PeterThoeny - 09 Oct 2004

We are very keen to incorporate the good functionality that exists in the many alternate competing systems but our challenge is to consolidate all this functionality into a common base so that both functionality and simplicity is available to all users. Without satisfying both of these it will fail to meet someone's needs and they will fork again. (Notable was YetAnotherNotificationSystem which primarily removed functionality to meet the aim of user interface simplicity).

So... the reason we start with MailerContrib is because the code exhibits a clean interface and comes with tests. It is also object oriented, so it exhibits LowCoupling and HighCoherence.

-- MartinCleaver - 05 Jan 2005

Overall goals

  • Consolidate multiple implementations to focus effort
  • Remain true to the system requirements (e.g. no external database)
  • Future goals: Coupled with a SubmitTopicByEmail implementation, this would provide round-trip email integration.

Benefits of Notification

  • provides users with feedback on topics they are interested in
  • when people get timely responses they participate more often

Nonfunctional requirements

  • An interface that is easy to understand and operate.

Features

Front end

  • When viewing a topic, a button to "subscribe me" - this saves the user the effort of remembering what the name of the topic that they needed was.
    • Avoiding problems with typos
    • Presents a more "professional" interface and makes TWiki look less of a "programmers hack"
    • Hides the implementation behind an interface so that the actually implementation can be changed later.
  • When saving a topic you could check boxes to:
    • Mail the next time this topic is edited (and only the next time)
    • Mail me immediately when this topic is edited (every time in the future)
  • A simple way to make WebNotify entries might be to use CommentPlugin UserTemplates. These could go in the WebBottomBar. This would need a UserSpecifiedRedirect so that subscribing would have you end up back at the topic, instead of WebNotify. Perhaps that would work with CommentPlugin, perhaps it needs it's own redirect parameter. -- SamHasler - 10 Oct 2004
  • A mechanism for the user to tell what topics they are subscribed to. This would be visible from the users home topic.
  • A mechanism to add, remove and suspend subscriptions
  • WebNotify syntax:
    • Why does the syntax of WebNotify not look like any other TWiki Plugins? i.e. (%xxx{...}%), so to allow extraction of any new parameter? -- PatrickNomblot 31 Dec 2004
    • MailNotificationEnhanced used the syntax: IN{...} / OUT{...} so that this could be aligned with macros standards. i.e. %IN{...} and %OUT{...}%. These might be are analysed only in WebNotify topics. What was important to Patrick was this made it possible to extract any parameter and allow more possibilites and facilitate evolution of mail notification.

Back end

  • Query to determine notifications of interest
  • Use of the DotMailnotify files to determine what has changed

Where to store subscription information

  • Subscription information storage could be
      • User-centric (in per-user subscription topic, see also UsernameMailnotify)
      • Web-centric (as currently in WebNotify
      • Topic-centric (in Topic text or in Meta Data)
    • Pros for User-Centric:
      • DONE Easy for the user to know what she is subscribed to,
      • ALERT! Subscription information in the user topic could clutter the topic
      • ALERT! Having a separate topic for subscriptions would cause topic proliferation in the web containing users so where there are large number of users, the WebNotify topic would grow very large.
      • DONEDONE Allow subscribing to write-protected webs (both web-centric and topic-centric methods deny this)
      • No code needed to prevent notifying about subscribes/unsubscribes
      • Access control can be used to ensure changes to a user's subscriptions is limited to authorised users
    • Pros for Web-centric:
      • DONE Does not require search through user topics to find subscriptions. (however, this is a minor point, as this process runs without user visibility)
    • Pros for Topic
      • For on-save methods the record of who to notify is easy to locate. This would be stored in the MetaData
      • perhaps the meta data ... could be used. -- GrantBow - 30 Jan 2003
    • Cons for Topic
      • Users would be prohibited from subscribing to topics in which they have no write access.

One method would be to have per-user notification lists that, when saved, update per-web/topic lists. This would allieviate having to scan every user-centric notify list for subscriptions every time every page is updated.

Although I agree that I like a per user view to show what I am subscribed to, I disagree that these should be stored on separate pages. Otherwise every notification page has to be read for every page save: on sites like TWiki.org this could be prohibitive. I'd suggest that per-page subscriptions should be stored on the page, e.g. the Interested Parties field.

-- MartinCleaver - 20 Dec 2004

Web-centric and user-centric suggestions require one to note the topic names and then correctly edit the WebNotify topic.

Has any thought been given to keeping notification requests totally separare from topics, ie in some form of backend database and to control subscriptions etc through * 'notify me' button on each topic * a per-user notification list which, script generated lists all current notification subscriptions and can be edited to be parsed again by a script which updates the database.

One main advantage would be that write access to a user's subscriptions could be limited to that user only as the script can perform the access verifications.

This method could be extended to have

  • whole web vs. listed topics subscription
  • immediate notification vs daily/weekly/other digest
  • once vs always notification
  • possibly notification on when certain condition met. Possible conditions I can envision:
    • grep finds a given word in the topic (nor not)
    • variable set in topic has certain value
    • some other status changes (eg support AskedQuestions vs AnsweredQuestions).

Immediate notification could possibly be rolled into the edit script which on final save could check whether any notification requests exist for the topic and take it from there.

Digest notification would check if any topic was newer than a user's last notification. Both would update a timestamp per registered user.

In case of once registration, the data of that notification would be recorded and thus prevent either script from notifying the user again until the timestamp is reset/removed.

-- MathiasKoerber - 01 Jan 2003

No, one doesn't need to write down and alter transcribe - possihbly with errors - while editing the WebNotify topic. That is IF there is a "button" that automates it.

What it boils down to is this:

Which method should the Cron Daeom use:

  1. Parse on the WebNotify topic for each web, with the extended syntax, or
  2. Parse the WebNotify file for backward compatibility AND each users Topic for their individual settings.

You can bet there will be some complications with thigns like redundancy.

As I've mentioned in UsingFormsForSettings, ther are some things that are better handled out of band. I don't particularly like the idea of a database as part of the core requirement for TWiki, I don't want to have to install some variant of SQL. The idea of keeping all the configuration information in a format that TWiki itself can access is a good one. The problem lies in the user interface. While storing everything as a Topic is a good idea, editing absolutely everything as a Topic is not a good idea. Hence the whole UsingFormsForSettings, something MegaTWiki does well.

This would have the advantage of

  1. Avoiding problems with typos
  2. Presents a more "professional" interface and makes TWiki look less of a "programmers hack"
  3. Hides the implementation behind an interface so that the actually implementation can be changed later.

The latter means that the actual implementation can vary in its syntax. It could be the extended WebNotify topic, it could be the individual user Topic or it could be, Votan forbid :-), a SQL database.

If this sounds like a classic recomendation from a 1970s book on structured design, well yes. Some ideas and principles are good and the new wave doesn't render them invalid.

-- AntonAylward - 02 Jan 2003

database in background does not have to be SQL behemoth. Simple CSV (comma-separated-values) might be OK, since flat files will be small, and we opem/parse many of them anyway. And we can use SQL syntax and upgrade to ie. MySQL later, if needed.

-- PeterMasiar - 02 Jan 2003

Think about the algorithm to be used for this feature. Do you want to forcably scan every user topic or every page topic? Should a separate set of perhaps straight text files held in non-topic text files be used instead as a "cache" so to speak.

I haven't looked at the patch yet, sorry. SQL should not be used per the TWikiMission statement, however perhaps the meta data abstraction feature could be used.

-- GrantBow - 30 Jan 2003

I comment in MetadataSucks that Perl has a DBI abstraction that makes it irrelvant whether the database is SQL, CSV or a remote ODBC.

-- AntonAylward - 31 Jan 2003

Settings

  • A DefaultNotificationPolicy preferences says whether the user wants to be notified about topics they have contributed to.
  • A DefaultNotificationTiming

Scope of notification

  • Existing WebNotify mechanism: notify about all topics in a web
  • by default Topics that a user has contributed to would have could add or take themselves off using a button "(Un)subscribe to topic".
  • Exclude certain topics on per-subscription basis
  • Exclude certain topics for all notifications (e.g., the Web* topics)
  • All topics that are new since last notification
  • Topics that have already been notified on:
    • the data of that notification would be recorded and so prevent from notifying the user again until the timestamp is reset/removed.
  • Exclude topics changed by user (NOTMINE)
  • notification on a complete topic tree (e.g. a MyTopic/** syntax to select child nodes)
  • Specific topics
  • Possibly notification on when certain condition met. Possible conditions:
    • grep finds a given word in the topic (nor not)
    • variable set in topic has certain value
    • some other status changes (eg support AskedQuestions vs AnsweredQuestions).

Identification of topic

  • Based on topic name
  • Based on topic content
  • Based on topic author or change author

Matching of topic

  • Strict (literal) text match
  • Regular expression based match, but has to be specifically keyed. E.g. use literal settings by default and have some kind of indicator (like, say, a "regex:" prefix) to allow people to use patterns.

Depth of match

  • Topic only
  • Descendents of topic to given depth (entire topic tree)

Delivery mechanisms

  • Notify by SMTP
  • Notify by RSS
  • Notify by Messenger (JABBER, MSN, Yahoo, AOL, etc.)
  • Notify to USEnet Newsgroup
  • Notify to a webpage log (e.g. per user)

Delivery timing and batches of changes

Changes are recorded chronologically in a change queue by Store's saveTopic method. Another process, (running continually or invoked very regularly) pushes out notifications according to the preference.

This means that notifications (even immediate ones) need not be performed by the saveTopic process, but rather by the global timer process. This moves notification to place where it can take as long as it needs. This decoupling speeds the save/view process for the editor.

Think about the algorithm to be used for this feature. Do you want to forcably scan every user topic or every page topic? Should a separate set of perhaps straight text files held in non-topic text files be used instead as a "cache" so to speak.

  • Notifications of changes:
    1. either executed in the view process of the user who made the change
    2. A reader, who thinks that somebody else may want to see a page
    • The writer who has just written a change who thinks that somebody else may want to see a page
    1. or by a external process
      • A cron job, running at some rate
      • An RSS Reader?
  • Notifications batched into Digest according to user preferences
    • as per a schedule (e.g. daily/weekly/monthly), a condition such as "x cumulative changes", or immediately
    • (implies extracting windows of changes according to each user's preference)
    • One implementation might be to check if any topic was newer than a user's last notification and to update a timestamp per registered user.
  • Notifications batched according to webpreferences
  • Notification process started by external means
    • cron * invoking bin script directly * invoking cgi-bin script via the web (e.g. wget)
    • Frequency * It is normal to batch updates to a couple of times a day, but more frequent it runs make for the smaller

One possibility would be to combine subscriptions stored in different places:

  1. WebNotify topic for each web,
  2. users Topic for their individual settings.

complications will arise from with issues such as redundancy.

Delivery consolidation

  • A single user-centric mail
  • Multiple web-centric mails

  • Are notifications combined? If so, how?
    • Messaging approaches, such as email, have a "once the message is sent, it is sent" aspect. They may delay a bit, and create a digest, but once the message is sent, it is gone.
    • RSS readers that assemble a web page do not necessarily bother the reader immediately, but are immediately readable. They allow, e.g., creating
      • "The 10 Most Interesting Things That Have Happened So Far Today" type of web pages. With "send and forget" messages like email, you can't do that without having a full day's worth of delay.
    • This seems to show that it is not a simple push/pull dichotomy. RSS used well seems to be "push to one place, but allow combining until the reader pulls".

Notification format

  • Single template
  • Multple user or web-specific templates

Open Questions

Generating subscription magic in a plugin

One comment on the suggestion by Peter to provide the link to the autoregnotify (or similar) via a plugin, I do not know how to modify templates via plugins, unless we put a hook in the existing template (as it is done with the change form button). The approach we use is that of an AddOnPackage. See the attached for an example, and also YetAnotherNotifySystem.

-- ThomasWeigert - 10 Oct 2004

I don't know how to modify templates without hooks via plugins either. Adding buttons to the view and edit templates is something I've wanted to do several times. -- CrawfordCurrie - 11 Oct 2004

Couldn't you just use CommentPlugin templates for subscribing in WebNotify and put them in the WebBottomBar instead of creating another plugin? You could even turn the tags in WebBottomBar into static HTML for speed.

You would probably want UserSpecifiedRedirect so you ended up back at the topic, instead of WebNotify. Although I'm not sure if that would work with CommentPlugin, perhaps it needs it's own redirect parameter.

-- SamHasler - 10 Oct 2004

Would my idea about a WebTopControlBar (PatternSkinDev) help? -- MartinCleaver

After we have consolidated as much as possible, is there still a need to have multiple systems?

Should we DecoupleFromTheCore this feature?

  • What PluginHandlers are needed to make this possible?
    • why not a more open and more plugin like interface to mail notification?
  • MailerContrib isn't a plugin yet, it's more like an add-on; are the needs of a mailer extension met by the plugins architecture, especially given the shortcomings of plugin speed overhead. I'd be interested in any proposals you have for a plugin mailer interface. I tried to implement MailerCOntrib with as narrow an interface to the core as possible, but if there is any way to improve on this it should be done. -- CrawfordCurrie - 02 Jan 2005

Is the notification period customisable on a per-user basis or is everyone mailed to at the same time

Where should delivery timing preferences be stored?

User documentation: installing a notification plugin should upgrade the help texts

Can we ensure that this mechanism supports all notification needs (e.g., action tracker, web changes, project tracker)

TWikiCommunity availability, wants and task-allocation

Richard, Patrick, Nicolas, Walter: I'd like to get the code merged as much as possible then split up so that you can work independently according to mutually-agreed interfaces.

  1. how much time can you give to this over the next week?
  2. which bits of the other implementations do you most like?
  3. which parts of your own code are you most and least attached to?

-- MartinCleaver - 12 May 2004

Unfortunately for TWiki, I'm getting rather tied up. I've been roped into taking a separate part time job, so I'm now working 40-50 hours a week, which is roughly 2x what I'm used to. Even though it's summer and I don't have classes to deal with, I'm still feeling a bit overwhelmed. So until I get used to the load I don't know how much time I'll be spending on TWiki stuff aside from what particularly catches my fancy. (Spending time on TWiki is probably unwise right now, and if I'm going to be unwise I might as well enjoy myself.)

-- WalterMundt - 17 May 2004


Refactored contributions from AntonAylward, MathiasKoerber, GrantBow, CrawfordCurrie, ThomasWeigert, MartinCleaver


To integrate:

Here "notification" may mean,

  • not just "Here's a change to a wiki page"
  • but also "Here's a wiki page that I (the browser) just found interesting.

BTW, my colleague Pieter Hartsook with some help from GrantBow figured out a way to generate an RSS feed for a single page. Embed a search for ^topicname$ in the call to WebRss, e.g. http://wiki.osafoundation.org/twiki/bin/view/Chandler/WebRss?skin=rss&web=all&scope=topic&search=^DevelopmentHome$

-- DuckySherwood - 23 Jul 2004

Notifying the user of changes

AntonAylward said in an email to Martin Cleaver:

> If someone hijacks an account the owner might not notice his password is invalid for some while.

One security measure might be to email the user a summary of their changes for the day. Especially after a period of inactivity. And any change to the user's form settings should be mailed immediately, to both a new and old email address.

Edit | Attach | Watch | Print version | History: r28 < r27 < r26 < r25 < r24 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r28 - 2007-09-16 - DavidWolfe
 
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.