Subject: KampalaMeetingLog2015x02x19.txt
Date: February 19, 2015 3:51:42 PM PST

[3:01pm]  HideyoImazu joined the chat room.
[3:01pm] PeterThoeny: hi hideyo-san
[3:01pm] HideyoImazu: Hi Peter
[3:02pm] PeterThoeny: i am right now at a beautiful place
[3:02pm] PeterThoeny: https://www.facebook.com/photo.php?fbid=10153016668668950
[3:02pm] PeterThoeny: online meeting in the sierra nevadas
[3:02pm] HideyoImazu: nice!
[3:02pm] HideyoImazu: I'm in the office earlier than usual
[3:03pm] PeterThoeny: the photo is kind of crappy from an automated HDR app on iphone
[3:03pm] PeterThoeny: the kids are skiing now
[3:03pm] HideyoImazu: not snowboarding?
[3:04pm] PeterThoeny: no, skiing
[3:04pm] PeterThoeny: my dauther likes snowboarding, but she is in a too warm place for winter sports (hong kong)
[3:04pm] PeterThoeny: happy lunar new year!
[3:05pm] HideyoImazu: happy lunar new year!
[3:05pm] PeterThoeny: what is new with you?
[3:06pm] HideyoImazu: I'm using a locally enhanced DatabasePlugin
[3:06pm] PeterThoeny: what additional feature?
[3:06pm] HideyoImazu: and I enhanced it further to support PostgreSQL
[3:06pm] PeterThoeny: ah
[3:06pm] HideyoImazu: putting a password is a big no-no
[3:07pm] PeterThoeny: how do you auth?
[3:07pm] HideyoImazu: and we use database name rather than server name to specify database
[3:08pm] HideyoImazu: all of our databases use kerberos authentication
[3:08pm] PeterThoeny: do you intend to release the db enhancements to the community?
[3:09pm] HideyoImazu: our database guys maintain the mapping between database name and server addresses
[3:09pm] HideyoImazu: that mapping is leveraged
[3:09pm] HideyoImazu: that's one item on my todo list
[3:10pm] HideyoImazu: to contribute to the twiki community
[3:10pm] PeterThoeny: nice!
[3:10pm] PeterThoeny: +10 min, shall we start?
[3:10pm] HideyoImazu: sure
[3:10pm] PeterThoeny: usual agenda
[3:10pm] PeterThoeny: http://twiki.org/cgi-bin/view/Codev/KampalaReleaseMeeting2015x02x19
[3:10pm] PeterThoeny: 1. Feature Requests for Kampala Release
[3:10pm] PeterThoeny: 2. Extensions
[3:10pm] PeterThoeny: 3. Review Urgent and Not So Urgent Bugs
[3:10pm] PeterThoeny: 4. Miscellaneous
[3:11pm] PeterThoeny: ---++ 1. Feature Requests for Kampala Release
[3:11pm] PeterThoeny: http://twiki.org/cgi-bin/view/Codev/TWikiFeatureProposals
[3:11pm] PeterThoeny: you posted a new one
[3:11pm] PeterThoeny: http://twiki.org/cgi-bin/view/Codev/SelfServiceMdrepoManagement
[3:11pm] PeterThoeny: Self-service mdrepo management
[3:11pm] PeterThoeny: looks good to me
[3:12pm] HideyoImazu: thanks
[3:12pm] PeterThoeny: quick accept
[3:12pm] PeterThoeny: +1 from me
[3:12pm] HideyoImazu: +1
[3:12pm] PeterThoeny: accepted by today's release meeting
[3:13pm] PeterThoeny: we talked about this before, it would be useful to have topic level caching of meta data 
for better performance
[3:13pm] PeterThoeny: the meta data repository would be a good choice
[3:13pm] PeterThoeny: for example, confluence shows the parent topic and list of children topics in the sidebar
[3:14pm] PeterThoeny: which is very useful for navigation
[3:14pm] PeterThoeny: technically it would be trivial for twiki to do the same
[3:14pm] PeterThoeny: but performance will suffer
[3:14pm] PeterThoeny: we can't have slower page load times
[3:15pm] PeterThoeny: caching key meta data of topics would solve this
[3:15pm] HideyoImazu: such as list of child topics?
[3:15pm] PeterThoeny: such as parent, children, backlinks, last update date, etc
[3:16pm] PeterThoeny: it would be cool to have other meta data cached in a database
[3:17pm] PeterThoeny: i am thinking stuff needed for query search, e.g. twiki form fields
[3:17pm] PeterThoeny: topic summary
[3:18pm] PeterThoeny: stuff that is typically needed in a query search
[3:18pm] PeterThoeny: enhancing the meta data repository is one option
[3:18pm] PeterThoeny: another one is a no-sql backend
[3:19pm] PeterThoeny: the no-sql approach would be a big indertaking
[3:19pm] PeterThoeny: "undertaking"
[3:19pm] PeterThoeny: how much effort do you see in using the meta data repo?
[3:19pm] HideyoImazu: my gut feeling is no-sql backend is cleaner
[3:20pm] HideyoImazu: it will be relatively easy to use mdrepo for caching
[3:20pm] PeterThoeny: yes, no-sql is very good in handling key-value pairs and hierarchies
[3:21pm] HideyoImazu: mdrepo consists of tables
[3:21pm] HideyoImazu: sites and webs tables by default
[3:22pm] HideyoImazu: a row of a table has a unique key and a set of key value pairs
[3:22pm] PeterThoeny: like adding a table for topics
[3:23pm] PeterThoeny: ok, something to ponder about
[3:23pm] PeterThoeny: any other feature you'd like to discuss?
[3:23pm] HideyoImazu: putting and getting data to/from mdrepo is easy
[3:24pm] HideyoImazu: but invalidating cached data properly might be a challenge
[3:24pm] HideyoImazu: with no-sql or sql backend, we can do atomic operation
[3:25pm] PeterThoeny: well, for a start it could be simply updating a table row for the topic on topic save
[3:25pm] PeterThoeny: so instead of saving children in topic we could do a dynamic lookup of topic's parents
[3:26pm] HideyoImazu: we will have multiple tables for caching ...
[3:26pm] PeterThoeny: since in database it will be fast
[3:26pm] HideyoImazu: a table for child topics
[3:26pm] PeterThoeny: it = children lookup
[3:26pm] HideyoImazu: a table for backlinks
[3:27pm] PeterThoeny: yes, if additional tables like for backlinks we need to have an intelligent invalidation & update
[3:27pm] HideyoImazu: a table in mdrepo doesn't have indexes
[3:27pm] HideyoImazu: a row is accessed by its key.
[3:28pm] HideyoImazu: on the webs table, each row represent a web
[3:28pm] PeterThoeny: it would be feasable to add indexes, such as for parent in topic table?
[3:28pm] HideyoImazu: and each row is accessed/identified by the web name
[3:29pm] HideyoImazu: at the bottom, each table in mdrepo is a set of key value pairs.
[3:30pm] HideyoImazu: the content of a table row (a set of key value pairs) is serialized to be put as a value
[3:30pm] PeterThoeny: time check: +30 min
[3:31pm] HideyoImazu: lib/TWiki/Mdrepo.pm is short
[3:31pm] PeterThoeny: ok
[3:31pm] HideyoImazu: if you read it, you see what I'm saying
[3:31pm] PeterThoeny: nothing urgent now, but useful
[3:31pm] PeterThoeny: let';s move on
[3:32pm] PeterThoeny: i have nothing else on features
[3:32pm] HideyoImazu: sure
[3:32pm] PeterThoeny: ---++ 2. Extensions
[3:32pm] PeterThoeny: i did some more updates on gpl v3 switch and 2015 y copyright
[3:32pm] PeterThoeny: all core plugins are done
[3:32pm] PeterThoeny: contribs are work in progress
[3:32pm] PeterThoeny: http://twiki.org/cgi-bin/view/Plugins/WebChanges
[3:33pm] PeterThoeny: the enhancement of the SetGetPlugin for JSON synatc is work in progress
[3:33pm] PeterThoeny: http://develop.twiki.org/~twiki/cgi-bin/view/Bugs/Item7611
[3:34pm] PeterThoeny: it is already possible to set a whole json structure to any level
[3:34pm] PeterThoeny: and to retrieve any part using a json path syntax
[3:34pm] PeterThoeny: the bug topic has some working examples
[3:35pm] PeterThoeny: setting json structures piece by piece is work in progress
[3:35pm] PeterThoeny: i can now set one level down, but not multiple levels
[3:35pm] PeterThoeny: docs are pending
[3:36pm] HideyoImazu: for what application will you use the feature?
[3:37pm] PeterThoeny: i plan to use it for an integration between twiki and jira using the jira rest api
[3:37pm] PeterThoeny: for this i need some manipulation of data structures
[3:37pm] PeterThoeny: that is where the json path comes in handy
[3:38pm] PeterThoeny: i also anticipate that this is useful for twiki apps in general
[3:38pm] PeterThoeny: do a search, store data in a specific way for easy reporting
[3:39pm] PeterThoeny: i already use spreadsheet hashes and array variables extensivly in complex twiki apps
[3:40pm] PeterThoeny: unlear yet is how to set a json object persistenly
[3:40pm] HideyoImazu: JSON data manipulation will be implemented in addition?
[3:40pm] PeterThoeny: the remember="on" syntax does not fit easily into the json syntax when you SET it
[3:40pm] PeterThoeny: not specifically
[3:41pm] PeterThoeny: more like retrieve aribtrary items (array item, object, complex structure) from a json object -- done
[3:41pm] PeterThoeny: using json path
[3:42pm] HideyoImazu: ah
[3:42pm] HideyoImazu: %GET{...}% can have a JSON path?
[3:42pm] PeterThoeny: and also set arbitrarry array items, objects, strings deep into a json structure
[3:42pm] PeterThoeny: -- work in progress
[3:42pm] PeterThoeny: yes GET already supports json path
[3:43pm] PeterThoeny: see examples in test cases
[3:43pm] PeterThoeny: simple example:
[3:43pm] PeterThoeny: %SET{ json_array = [ "a", "b", "c" ] }%
[3:43pm] PeterThoeny:  %GET{ json_array[1] }% -- will return "b"
[3:43pm] PeterThoeny: (without quotes)
[3:44pm] PeterThoeny: thsi works to any depth, such as  %GET{ foo[1].bar.moo[2][5] }%
[3:45pm] PeterThoeny: let's move on
[3:45pm] PeterThoeny: ---++ 4. Miscellaneous
[3:45pm] PeterThoeny: sorry, before that
[3:45pm] PeterThoeny: ---++ 3. Review Urgent and Not So Urgent Bugs
[3:45pm] PeterThoeny: http://develop.twiki.org/~twiki/cgi-bin/view/Bugs/WebChanges
[3:46pm] PeterThoeny: i don;t have anything
[3:46pm] PeterThoeny: do you?
[3:46pm] HideyoImazu: me either
[3:46pm] PeterThoeny: ---++ 4. Miscellaneous
[3:46pm] PeterThoeny: anything?
[3:47pm] PeterThoeny: we should discuss the develop.twiki.org server switch
[3:47pm] HideyoImazu: no
[3:47pm] PeterThoeny: it is mostly done
[3:47pm] PeterThoeny: pending:
[3:47pm] PeterThoeny: - sync .htpassd from twiki.org to develop.twiki.org
[3:47pm] PeterThoeny: - backup develop.twiki.org to twiki.org backup drive
[3:47pm] HideyoImazu: and trac
[3:48pm] PeterThoeny: - get trac installed for web based diffs
[3:48pm] PeterThoeny: all other stuff should be working
[3:48pm] PeterThoeny: if you find time could you look into trac?
[3:48pm] PeterThoeny: i will do the sync and backup
[3:49pm] HideyoImazu: sure
[3:49pm] PeterThoeny: thank you
[3:49pm] PeterThoeny: this is all i had
[3:49pm] PeterThoeny: shall we close the meeting?
[3:49pm] HideyoImazu: sure
[3:50pm] PeterThoeny: ok, thank you hideyo-san!
[3:50pm] PeterThoeny: i'll post the logs and minutes
[3:50pm] PeterThoeny: ttyl!
[3:50pm] HideyoImazu: ttyl!