Feature Proposal: RESTful Interface for Plugins

Motivation

To provide integration with external systems (or even a decoupled integration between TWikiApplications), it's useful to provide a way to "invoke" Plugins methods via a URL.

Also, this is the easiest way to implement AJAX based interfaces using TWiki plugins.

Description

A new script in bin called rest that can be invoked via http in a similar way as the view script (see Examples, bellow) to execute a plugin method. It'll print the result directly to the stream unless the endPoint parameter is supplied, in which case the control is redirected to the given topic.

The rest script can receive one parameter:

Any additional parameter is passed directly to the method (i.e: The method can get any other parameter using the $query object)

Impact and Available Solutions

Note: Commited to DEVELOPBranch r4665

Note: Created Bugs:Item116

Documentation

If necessary, user documentation of new features introduced by this proposal.

Examples

http://my.host/bin/rest/EmptyPlugin/testRest

Will invoke TWiki::Plugin::EmptyPlugin::testRest, and print the result directly to the stream.

http://my.host/bin/rest/EmptyPlugin&method=testRest

Will invoke TWiki::Plugin::EmptyPlugin::testRest, and redirect the control to SomeWeb.SomeTopic

Implementation

Security Issues

For security reasons, two checks are performed on the parameters:

• The name of the plugin must be checked to be valid WikiWord.
• The method must not contain illegal characters.

Discussion:

My primary concern for this is security. Are those two checks enough?

-- RafaelAlvarez - 05 Jul 2005

personally i'd like some sort of twiki-based configurable security settings..

-- SvenDowideit - 06 Jul 2005

What kind of checks do you propose?

-- RafaelAlvarez - 06 Jul 2005

notice how i very carefully left that up to you? thus avoiding claiming that the TWikiGroups would an important part of it - .htaccess can only be configured by the httpd admin, so more granularity is needed...

-- SvenDowideit - 07 Jul 2005

Ahh... THAT kind of security! We're talking about two different things

My question was more in the "Code security" lines (not leaving any door open for exploits).

As for the "User security", I was thinking to leave the responsability to decide "who can do what" to the plugin. See, I have envisioned two main uses for this interface:

• Prevent the plugins from having to distribute a script in the bin directory, by just adding a method to the plugin code and calling the rest script. In this case, the user is only "known" if login is needed to perform any operation
• External apps querying a TWikiApplication for information. In this case, the user is not known and can't be known unless the client implements whatever is required to autenticate using the http mechanism.

In both cases, it should be up to the plugin to decide if the request is allowed or not.

-- RafaelAlvarez - 07 Jul 2005

i'm not sure that you should put so much trust into the plugin (or worse, such an implementation burden) stuff like user security needs to be done in the core parts, so that plugins can concentrate on doing whatever they do..(I think)

-- SvenDowideit - 08 Jul 2005

The thing is that the Core can authenticate the user (he is who he says he is, and belogs to this and that group) but cannot validate if the user is allowed to perform an specific operation as defined by the plugin. That is, the core performs the authentification and the plugin must perform the authorization, ideally leveraging the current TWikiAccessControl mechanism.

For example, our local version of the XpTrackerPlugin allows the creation of new stories only to users that have write access to the iteration topic.

-- RafaelAlvarez - 08 Jul 2005

yeah, but that restriction (should really) be due to the core refusing to allow that users session to save to that topic - its not like the XpTrackerPlugin should be able to decide (ie over-ride) the TWikiAccessControl - similarly, its probably not up to the Plugin to decide if it will allow a user to use it via this new script, rather, the core security code should decide (and thus control & set) the plugin's ablility to do things.

but you know - this is probably too blue sky considering the current state of security in twiki - where a plugin is trusted to abide by the rules.

-- SvenDowideit - 08 Jul 2005

hmm.. that gave an idea.

Suppose that the method parameter is changed to an pluginaction parameter. The plugin should have a hook (like dispatchPluginAction($session,$action) or something like that) that knows what method to call.

The interesting bit is how this allows to configure security. The core could check the plugin topic (or WebPreferences or TWikiPreferences) for settings in the form ALLOW<uc($action)> and DENY<uc($action)>, much in the same way as the ALLOWVIEWTOPIC and DENYVIEWTOPIC settings work, and use that to allow/deny the execution of the action in the plugin.

For example, calling the URL

http://my.host/bin/rest/SomeWeb/WebHome?plugin=EmptyPlugin&action=test

would force the core to check for the setting ALLOWTEST and DENYTEST in the EmptyPlugin topic, or EMPTYPLUGIN_ALLOWTEST and EMPTYPLUGIN_DENYTEST in the WebPreferences or TWikiPreferences.

Is this more like you had in mind?

-- RafaelAlvarez - 08 Jul 2005

Can you describe a scenario in which this would be used? I mean the whole interface, not just the security.

-- CrawfordCurrie - 09 Jul 2005

think xmlrpc / soap interface, or a lighterweight rss feed ? (i hope i've interpreted your intentions correctly)

-- SvenDowideit - 09 Jul 2005

I had two goals in mind for suggesting this feature:

• Remove the need for plugins to publish scripts in the "bin" directory. That way they can leverage the "standard" script infrastructure as provided by UiDotPm (ie: Security and the CLI processing when calling from a cron job or manually)
• Provide a way to "talk" to plugins without the need to pass througth a topic.

The second goal includes the ability to have xmlrpc/SOAP/whatever interfaces, and RSS plugin (thus removing the hack in the core). So yes, you interpreted my intentions correctly

But let me describe a (realworld) scenario. We have a project tracking web at work. I need to be able to:

• "publish" to that web the results of the automated build:
  • Change the status of all the tickets that passed acceptance testing.
  • Mark the ticket (or tickets) just commited as "Deployed".
  • Mark those ticket just commited that failed the acceptance.
• Check "who's next" in the integration queue and notify him.
• Integrate the tracking web with the IDE so it's easier to keep track of the time spent working on an issue.

-- RafaelAlvarez - 10 Jul 2005

bloody brilliant! - that dovetails neatly with my desire that code based addons become the same as plugins..

-- SvenDowideit - 11 Jul 2005

Thanks Rafael. I was going to whinge until I got to the last bullet in your list. That is something that I can appreciate as being very valuable. It was to support that kind of application that I did the MailInContrib.

-- CrawfordCurrie - 11 Jul 2005

I changed the syntax so it reflect the philosophy that the URL is the command, and the parameters are, well..., the parameters.

-- RafaelAlvarez - 11 Jul 2005

I'll commit the rest script and supporting modules to DakarRelease, and wait for EdinburghRelease (where AccessControlLists should be implemented) for the security stuff.

-- RafaelAlvarez - 13 Jul 2005

Are there any examples of plugins using REST?

-- ArthurClemens - 01 Jun 2006

yep grep -i resthandler /lib/TWiki/Plugins/.pm in twikiplugins shows

AddMetaPlugin/lib/TWiki/Plugins/AddMetaPlugin.pm:    TWiki::Func::registerRESTHandler('example', \&restExample);
EmptyPlugin/lib/TWiki/Plugins/EmptyPlugin.pm:    # and register a RESTHandler. (remove code you do not need)
EmptyPlugin/lib/TWiki/Plugins/EmptyPlugin.pm:    TWiki::Func::registerRESTHandler('example', \&restExample);
GenerateSearchPlugin/lib/TWiki/Plugins/GenerateSearchPlugin.pm:    TWiki::Func::registerRESTHandler('search', \&restSearch);
GnuPlotPlugin/lib/TWiki/Plugins/GnuPlotPlugin.pm:    TWiki::Func::registerRESTHandler('gnuPlot', \&restGnuPlot);
PloticusPlugin/lib/TWiki/Plugins/PloticusPlugin.pm:    TWiki::Func::registerRESTHandler('Ploticus', \&restPloticus);
TimeTablePlugin/lib/TWiki/Plugins/TimeTablePlugin.pm:    ####TWiki::Func::registerRESTHandler('example', \&restExample);

there are more, but those, you can look at now

-- SvenDowideit - 01 Jun 2006

Hello,

Please help or guide me towards the following:

How do I communicate with Twiki from other system (i.e. built with PHP)? For example: I am running a site with members. I want to sync or transfer the members data one the following way:

1. Existing users from my site's database to Twiki user database (bypassing the email verification, as users are already verified at my site).

2. New user sign-up in my site, automatically adds to Twiki (bypassing the email verification, as users will be verified from my site).

Thank you!

-- AnupamSaha - 2010-12-20

Above question is a cross-post of Support.SID-01060.

-- PeterThoeny - 2010-12-20