Subject: KampalaMeetingLog2013x10x24.txt
Date: October 24, 2013 5:28:53 PM PDT

[2:55pm]  HideyoImazu joined the chat room.
[3:01pm] PeterThoeny: hi hideyo-san
[3:02pm] HideyoImazu: hi Peter
[3:02pm] PeterThoeny: bra, getting cold here
[3:02pm] PeterThoeny: bra that is
[3:02pm] PeterThoeny: brr that is
[3:02pm] PeterThoeny: dawn autocorrection
[3:02pm] HideyoImazu: not so cold here yet
[3:02pm] PeterThoeny: darn auto correction
[3:03pm] HideyoImazu: i still don't need a jacket. though most people wear it these days.
[3:03pm] PeterThoeny: took gloves riding the bicycle
[3:04pm] PeterThoeny: cupertino currently has 13 degrees c
[3:04pm]  two- joined the chat room. (67.171.131.23)
[3:04pm] PeterThoeny: hi two- you are number three here 
[3:05pm] two-: 
[3:05pm] HideyoImazu: it's 17 or so here, i suppose
[3:05pm] PeterThoeny: two-:  can you give a quick intro? how use twiki etc
[3:06pm] •PeterThoeny founder of twiki, focus on billing a better workplace
[3:06pm] PeterThoeny: "building a better workplace"
[3:07pm] two-: twiki is a good research tool. i have it setup on local user with nginx+perlbrew, and i use 
the rss feature a lot to keep track of changes
[3:07pm] PeterThoeny: haha, am i distracted or is this just auto-correction issue?
[3:08pm] PeterThoeny: was it difficult to setup on nginx?
[3:08pm] HideyoImazu: i'm taking care of a twiki installation in Morgan Stanley having 8000+ webs and 800k+
topics
[3:08pm] PeterThoeny: hideyo-san's team also made the open source twiki scale to that scale 
[3:09pm] PeterThoeny: meeting at http://twiki.org/cgi-bin/view/Codev/KampalaReleaseMeeting2013x10x24
[3:09pm] PeterThoeny: agenda:
[3:09pm] PeterThoeny: 1. TWiki-6.0 Release Post-Mortem
[3:09pm] PeterThoeny: 2. Release Focus of Next Release
[3:09pm] PeterThoeny: 3. Feature Requests for Jerusalem Release
[3:09pm] PeterThoeny: 4. Review Urgent and Not So Urgent Bugs
[3:09pm] PeterThoeny: 5. Extensions
[3:09pm] PeterThoeny: 6. Miscellaneous
[3:10pm] PeterThoeny: two-:  do you have a twiki.org account? wikiname?
[3:10pm] PeterThoeny: ---++ 1. TWiki-6.0 Release Post-Mortem
[3:10pm] PeterThoeny: release went really well i think
[3:10pm] PeterThoeny: no issues
[3:10pm] PeterThoeny: no new bugs so far
[3:10pm] PeterThoeny: congrats all!
[3:11pm] HideyoImazu: glad to hear that
[3:11pm] PeterThoeny: outreach is also all done
[3:11pm] HideyoImazu: but I realized that tagmeplugin has several issues in its code and documentation
[3:12pm] PeterThoeny: we got mentions by two german linux magazines
[3:12pm] PeterThoeny: how fully we get some more mentions
[3:12pm] HideyoImazu: i've been working on it for a few days and soon i will commit to the repository
[3:12pm] PeterThoeny: i'll keep track at http://twiki.org/cgi-bin/view/Codev/TWikiInTheNews
[3:12pm] two-: i was new to nginx and i had no idea what was involved, but iirc all that you need to do is 
1. configure nginx.conf with proper server{} section pointing to your http dir; 2. move the twiki dir into
it; 3. installed these modules: local::lib, CGI-Session-4.48, FreezeThaw-0.5001, Time-modules-2013, 
GD-2.50, Rcs-1.05; 4. install fcgi-wrapper and spawn-fcgi. 5. create script with case--switch statements 
to manage launch/reset; 6. and
[3:12pm] two-:  login to nginx
[3:13pm] PeterThoeny: thanks hideyo-san for working on tagmeplugin!
[3:14pm] PeterThoeny: nice two-! i invite you to update http://twiki.org/cgi-bin/view/Codev/TWikiOnNginx 
with your additional info!
[3:14pm] PeterThoeny: that way other folks can benefit from this ultrafast webserver engine
[3:16pm] HideyoImazu: The issues I found out with TagMePlugin are described on Item7375. besides Item6816
[3:16pm] PeterThoeny: two-:  for security maybe better not to have whole twiki tree in http
[3:16pm] PeterThoeny: all you need is twiki/pub in http, and twiki/bin as cgi
[3:18pm] PeterThoeny: if whole twiki tree is http anybody can access page data disregarding access control,
e.g. twiki/data/Anyweb/AnyTopic.txt
[3:18pm] PeterThoeny: one item is pending on 6.0.0 release: build twiki-vm image
[3:18pm] PeterThoeny: i started, hopefully i can finish this weekend
[3:19pm] PeterThoeny: it takes some more work to create new centos 6 image to run twiki
[3:19pm] HideyoImazu: cool
[3:19pm] PeterThoeny: current twiki-vm is on centos 5
[3:20pm] PeterThoeny: as for post-mortem, one thing i'd like to have addressed: distribute the work more 
among community
[3:21pm] PeterThoeny: this month i spent so much on unbillable open source work that my paycheck will be 
quite small this month (my wife will kill me)
[3:21pm] HideyoImazu: understood
[3:21pm] PeterThoeny: besides just building the release there is a lot of work on updating twiki.org and
develop.twiki.org content
[3:22pm] PeterThoeny: according to stats, this month i updated 1000+ topics on twiki.org and 600+ on
develop.twiki.org
[3:22pm] HideyoImazu: wow!
[3:22pm] PeterThoeny: that comes back to the question of building up the community
[3:23pm] PeterThoeny: for example, on develop.twiki.org all features and bug fix topics had to be changed 
from "waiting for release" to "closed"
[3:24pm] PeterThoeny: on twiki.org all accepted but unfinished feature proposals had to be changed from
jerusalem to kampala
[3:24pm] PeterThoeny: all reports in plugins web, support web, codec web had to be updated to add 6.0
[3:24pm] PeterThoeny: etc etc
[3:25pm] PeterThoeny: the long checklist is at http://twiki.org/cgi-bin/view/Codev/ProductionReleaseChecklist
[3:25pm] HideyoImazu: there should a person in charge of those
[3:25pm] PeterThoeny: when george was the release manager he did the actual build and i did the checklist stuff
[3:26pm] PeterThoeny: this time i did both
[3:26pm] PeterThoeny: frankly, this month was a bit on the limit for me
[3:26pm] PeterThoeny: especially with customer waiting
[3:27pm] PeterThoeny: this is just a fyi
[3:27pm] PeterThoeny: key point is to attract more contributors
[3:27pm] HideyoImazu: sure
[3:28pm] PeterThoeny: ---++ 2. Release Focus of Next Release
[3:28pm] PeterThoeny: i suggest we brainstorm today on release focus, possibly use the rest of the hour
[3:29pm] PeterThoeny: e.g. skip other agenda items, what do you think?
[3:29pm] HideyoImazu: i'm for it
[3:29pm] PeterThoeny: ok
[3:29pm] PeterThoeny: let's first look at previous jerusalem release focus:
[3:29pm] PeterThoeny: http://twiki.org/cgi-bin/view/Codev/JerusalemRelease#Release_Focus
[3:30pm] HideyoImazu: this may not be a focus but I plan to work on making dynamic access control more robust
[3:30pm] PeterThoeny: new kampala release focus will be recorded at:
[3:30pm] PeterThoeny: http://twiki.org/cgi-bin/view/Codev/KampalaRelease#Release_Focus
[3:30pm] PeterThoeny: good
[3:31pm] PeterThoeny: my personal goal for previous release that did not make it is: twiki app packaging standard
[3:31pm] PeterThoeny: so this is a release focus for me for kampala release
[3:31pm] PeterThoeny: i think this is very important for the twiki project
[3:32pm] PeterThoeny: we should brainstorm a bit on three items: usability, performance, app platform
[3:33pm] PeterThoeny: first on better usability
[3:33pm] PeterThoeny: what comes to mind?
[3:34pm] HideyoImazu: still, twiki doesn't seem to appeal to non technical people
[3:34pm] HideyoImazu: at least around me
[3:35pm] HideyoImazu: it's cumbersome to put an inline image on a topic
[3:35pm] HideyoImazu: compared to paste an image on Microsoft Word document
[3:36pm] HideyoImazu: and these days, you can attach a file to Gmail on browser by drag and drop
[3:36pm] PeterThoeny: ideally it should be possible to drag and drop an image into the wysiwyg editor
[3:36pm] PeterThoeny: and in attach screen, a drag & drop should be possible too
[3:37pm] PeterThoeny: not sure if the latest tinymce 4 supports drag and drop of images
[3:37pm] PeterThoeny: another usability thing:
[3:38pm] PeterThoeny: evaluators are typically administrators and technical
[3:38pm] PeterThoeny: they expect point and click administration
[3:38pm] HideyoImazu: i'm doing some research on editing a topic with MS Word
[3:38pm] PeterThoeny: we have now relatively good user management
[3:39pm] PeterThoeny: but preferences settings and access control are still a bit too difficult
[3:39pm] PeterThoeny: so that is an area we should focus on i think
[3:39pm] PeterThoeny: i might work on improving the preferencesplugin
[3:40pm] PeterThoeny: have have some ideas to do that for intuitive operation using ajax
[3:40pm] PeterThoeny: and jquery
[3:41pm] PeterThoeny: one other point on usability:  google docs are pretty much the standard/expectation nowadays
[3:41pm] PeterThoeny: that is, we should strive to have at least as good capabilities
[3:42pm] PeterThoeny: i think on wysiwyg editor we will get there by using latest tinyce 4
[3:42pm] PeterThoeny: missing gap: calendar and spreadsheet
[3:43pm] PeterThoeny: on calendar, may be a simple integration with google calendar is enough?
[3:43pm] HideyoImazu: can't we get rid of the round trip of tinymce retrieving the page content?
[3:44pm] PeterThoeny: good point
[3:44pm] PeterThoeny: i do no understand why the tml->html conversion could not be done before the editor is loaded
[3:44pm] HideyoImazu: some operation teams are using CalendarPlugin to show on-call schedule and team members'
vacation schedule
[3:45pm] HideyoImazu: WysiwygPlugin without TinyMCEPlugin doesn't have that extra round trip, I understand.
[3:45pm] PeterThoeny: yes, google calendar and twiki calendar are two overlapping circles
[3:46pm] PeterThoeny: so one option is to prettify twiki calendar and make it more usable
[3:46pm] PeterThoeny: with point & click events, more views (not just monthly) etc
[3:47pm] HideyoImazu: I wonder if it's possible to export ical data or something alike from CalendarPlugin
[3:47pm] PeterThoeny: that is probably very easy to do
[3:48pm] PeterThoeny: you can easily format a search, and you can set the mime content-type
[3:48pm] HideyoImazu: SharePoint can do it so that Outlook can display a SharePoint calendar
[3:48pm] PeterThoeny: likely similar to csv export i blogged about
[3:49pm] PeterThoeny: http://twiki.org/cgi-bin/view/Blog/BlogEntry201111x2
[3:50pm] PeterThoeny: on spreadsheet, we have all the power and more
[3:50pm] PeterThoeny: but not the usability
[3:50pm] PeterThoeny: ideally we have a sexy looking spreadsheet like google docs or excel
[3:51pm] PeterThoeny: that means some combination of edittableplugin and spreadsheetplugin where a lot of the
magic happens in javascript
[3:51pm] HideyoImazu: an integration with TableEditPlugin?
[3:52pm] PeterThoeny: well, not sure on best approach to package - new plugin? enhance edittableplugin?
[3:53pm] HideyoImazu: oops EditTablePlugin
[3:53pm] PeterThoeny: as far as fronted magic, the DataTables jquery plugin looks very suitable
[3:53pm] PeterThoeny: https://datatables.net/
[3:53pm] PeterThoeny: "frontend magic"
[3:55pm] HideyoImazu: i'll take a look
[3:55pm] PeterThoeny: anything else on usability?
[3:56pm] PeterThoeny: if not, move on to: performance
[3:57pm] HideyoImazu: ok
[3:57pm] PeterThoeny: i think the metadata repository is promising
[3:57pm] PeterThoeny: could this be used not just for large sites
[3:57pm] PeterThoeny: ?
[3:57pm] PeterThoeny: here is an idea
[3:58pm] PeterThoeny: use metadata repository as a cache for twiki topics:
[3:58pm] PeterThoeny: - topic meta
[3:58pm] PeterThoeny: - form meta
[3:58pm] PeterThoeny: - attachment meta
[3:58pm] PeterThoeny: - topic summary
[3:58pm] PeterThoeny: - headings cache
[3:59pm] HideyoImazu: ah
[3:59pm] PeterThoeny: - wikiword links cache
[3:59pm] PeterThoeny: that way search could be made much faster by making it made aware of the metadata repository
[4:00pm] PeterThoeny: the idea is to have this optional
[4:00pm] PeterThoeny: e.g. turn on will start caching meta data
[4:00pm] PeterThoeny: e.g. if turned on and copying a web from backup or another twiki, start caching meta data
[4:00pm] PeterThoeny: you get the idea
[4:01pm] PeterThoeny: so, rename with backlinks check could be very fast
[4:01pm] HideyoImazu: yes. metadata repository is a collection of nonvolatile hashes - so you can do those things
[4:01pm] PeterThoeny: turning on wikiwords with tooltip info could be very fast
[4:02pm] PeterThoeny: twiki form queries could be very fast
[4:02pm] PeterThoeny: etc
[4:02pm] PeterThoeny: that is the metadata repository approach
[4:03pm] HideyoImazu: as of now there are only a couple of metadata files in the repository
[4:04pm] HideyoImazu: if your idea is implemented, a lot more metadata files
[4:05pm] PeterThoeny: another approach is to use a new (optional) backend: nosql or object database
[4:06pm] PeterThoeny: if more matadata files: watch out for number of files
[4:06pm] PeterThoeny: needs to be less than 1:1 to number of topics
[4:07pm] PeterThoeny: update is much less frequent than view
[4:07pm] HideyoImazu: sure
[4:07pm] PeterThoeny: so repository should be optimized for view
[4:07pm] HideyoImazu: there will a certain number of metadata files per web
[4:07pm] PeterThoeny: yes, that is scalable
[4:08pm] PeterThoeny: so, topic save would need to update metadata files of that web, with lock to get
transactional behavior
[4:08pm] PeterThoeny: time check: +68 min
[4:09pm] HideyoImazu: atomicity of a operation might be an issue
[4:09pm] PeterThoeny: how much longer?
[4:09pm] HideyoImazu: i can continue at least by 9 am, which is 51 min away
[4:10pm] PeterThoeny: well, if implemented that topic save locks metadata during save, other saves in that web
need to wait
[4:10pm] PeterThoeny: which is a non-issue during save
[4:10pm] PeterThoeny: waiting a few .1 sec more on wait is ok
[4:11pm] PeterThoeny: i should stop latest in 20 min
[4:11pm] HideyoImazu: locking is fine. but what about atomicity of metadata update?
[4:12pm] HideyoImazu: a topic update ends up updating multiple metadata files.
[4:12pm] PeterThoeny: if need to you update several files you need to have external locking during whole
transaction
[4:12pm] HideyoImazu: if something crashes in the middle of metadata update, a partial update of metadata may happen
[4:13pm] PeterThoeny: yes, that can be an issue
[4:14pm] PeterThoeny: not just crash, a simple esc or browser-back by the user on page save does stop the save
operation
[4:14pm] HideyoImazu: also, during metadata update, metadata reading needs to be blocked so that inconsistent
metadata won't be read.
[4:15pm] PeterThoeny: that might be more forgiving
[4:15pm] PeterThoeny: twiki is not a bank transaction, so minor inconsistencies on page display should be fine?
[4:15pm] HideyoImazu: not necessarily
[4:16pm] PeterThoeny: but agreed on transactional metadata update
[4:16pm] HideyoImazu: we have web metadata in the repository
[4:16pm] PeterThoeny: one possible solution:
[4:16pm] HideyoImazu: and we have three twiki site running in a federation
[4:17pm] HideyoImazu: when a web metadata is updated, it needs to be propagated to the others in the federation
[4:17pm] HideyoImazu: at first, I didn't do anything special about metadata propagation
[4:18pm] PeterThoeny: add a deamon that does the metadata handling: deamon gets a call with all instructions
for update, and does the update asynchronously
[4:18pm] HideyoImazu: then I realized that at times, twiki behaved strangely for a brief period of time
[4:18pm] PeterThoeny: that way, if script is stopped we are good
[4:18pm] PeterThoeny: this does not solve the issue of server crash./restart
[4:18pm] HideyoImazu: that was happening when metadata files are being copied.
[4:18pm] PeterThoeny: but that is very unlikely
[4:19pm] HideyoImazu: twiki was reading incomplete data.
[4:19pm] HideyoImazu: if you have 10M+ page views per month, that does happen
[4:19pm] PeterThoeny: ah, yes, that makes sense now on lock during read
[4:20pm] HideyoImazu: so now, when metadata files are copied, they are copied to a temporary directory
[4:20pm] PeterThoeny: makes sense
[4:20pm] HideyoImazu: the metadata file directory twiki is seeing is actually a symbolic link
[4:21pm] HideyoImazu: after file copying completes, the symbolic link is changed to the new direcotry
[4:21pm] PeterThoeny: that is fast
[4:22pm] PeterThoeny: with current metadata, what happens if you manually copy new topics to a web? how are
they detected?
[4:22pm] HideyoImazu: we don't have topic metadata in the repository
[4:22pm] HideyoImazu: only web and site metadata
[4:23pm] PeterThoeny: ah, so this is currently a non-isue
[4:23pm] PeterThoeny: "issue"
[4:23pm] HideyoImazu: right
[4:23pm] PeterThoeny: how feasible is a topic metadata cache with your current technology?
[4:25pm] PeterThoeny: two-:  still here? what is your twiki.org wikiword?
[4:26pm] HideyoImazu: it needs a good amount of thinking, experiment, and coding
[4:26pm] HideyoImazu: according to my gut feeling
[4:26pm] PeterThoeny: yes, it's a major undertaking
[4:27pm] HideyoImazu: using a database having atomic transaction makes things easier
[4:27pm] PeterThoeny: needs to be solid, such as recache where needed (admin adds topics on file level, ...)
[4:30pm] PeterThoeny: time check: +90 min
[4:30pm] PeterThoeny: i think we hold table the app platform feature for another time
[4:30pm] PeterThoeny: so for other agenda items
[4:30pm] PeterThoeny: good brainstorming!
[4:30pm] HideyoImazu: agreed
[4:30pm] PeterThoeny: as a summary:  focus
[4:31pm] PeterThoeny: - better app platform
[4:31pm] PeterThoeny: - usability
[4:31pm] PeterThoeny: - performance
[4:31pm] PeterThoeny: shall we close the meeting?
[4:31pm]  two- left the chat room. ("Textual IRC Client: www.textualapp.com")
[4:32pm] HideyoImazu: yes
[4:33pm] PeterThoeny: i'll post the logs
[4:33pm] PeterThoeny: thanks HideyoImazu-san, was a good meeting!
[4:34pm] HideyoImazu: thanks, peter. ttyl
[4:34pm] PeterThoeny: ttyl
[4:40pm]  HideyoImazu left the chat room. (Ping timeout: 250 seconds)