Session Start: Mon May 21 21:59:24 2007 Session Ident: #twiki_release [21:59] * Now talking in #twiki_release [21:59] * Topic is 'http://twiki.org/cgi-bin/view/Codev/FreetownReleaseMeeting2007x05x07' [21:59] * Set by wnorris on Mon May 07 17:39:48 [21:59] Good evening / afternoon all [22:00] * ktwilight has joined #twiki_release [22:00] * ArthurClemens has joined #twiki_release [22:01] * OliverKrueger has joined #twiki_release [22:01] HelloWorld [22:01] HelloTWiki [22:01] nice to see you here oli [22:01] hi [22:02] holla - ish - need to make dinner too [22:02] hi arthur, crawford, kwang, kenneth, steve, sven and will! [22:02] nice turnout today [22:02] hey! [22:02] and in time :-) [22:02] 'lo [22:02] Hi Peter, Arthur, Steve,... [22:02] hey there [22:03] howdy [22:03] release topic at http://twiki.org/cgi-bin/view/Codev/FreetownReleaseMeeting2007x05x21 [22:04] * Lavr__ changes topic to 'http://twiki.org/cgi-bin/view/Codev/FreetownReleaseMeeting2007x05x21' [22:04] i think we can start [22:04] who is taking the minutes? [22:04] kenneth, anybody else? [22:04] I am ready as always [22:04] thanks [22:04] thanks for that [22:04] who is facilitating? [22:04] pth, anybody else? [22:05] hmm? [22:05] i go for it [22:06] proposed agenda: [22:06] # 1. Action Item Review [22:06] # 2. Review Proposed Features [22:06] # 3. Release Schedule of TWiki Release 4.2 [22:06] usual agenda [22:06] anything to add? [22:06] oh, btw, found out who was DOSing us the last week, and they got hit by their university network admins [22:06] vcu library... [22:07] ok, lets start [22:07] ---+ 1. Action Item Review [22:07] # All Proposers with accepted proposals: Update ProposedFor field in accepted proposals (not yet implemented) to GeorgetownRelease if they will not be ready for feature freeze date. [22:07] kenneth, did you monitor this? status? [22:08] I have not seen many changes for Georgetown. I have changed a few based on what I think will happen [22:08] please push all my stuff other than the usermapper changes to next release [22:08] there goes our search results pagination :-( [22:08] i've now had 6 days of (d|w).t.o server pain [22:09] ArthurClemens: because sven was the implementer? [22:09] yes [22:09] the pagination is useful [22:09] no argument there [22:09] anyone here who could take this implementation? [22:10] nope, sry. [22:10] We have 14 all good accepted proposals waiting to be implemented. Some will have to be deferred or 4.2 needs to be pushed quite far out. [22:10] arthur: could you state that on the proposal topic that you are looking for an implementer? [22:11] i'll look *into* it today [22:11] ok [22:11] i'd like to defer all mine, so i can focus on making the mapper stuff solid [22:11] very cool! [22:11] no commitment, but i'll take a look and see how much time, etc [22:11] understood [22:11] i'm testing another change atm [22:11] and get back to you, updating the release page, i guess [22:12] ok, cool [22:12] ok, next [22:12] # Kenneth: UseIsoDates: Kenneth takes the action to re-write the proposal for a renewed decision round. [22:12] marked as done [22:12] Yes. [22:12] kenneth? [22:12] Done. [22:13] # Sven: TopicCaseSensitivity: Sven to clarify the spec [22:13] And I think I can implement it myself if we go for my proposal and delay freeze by a week [22:13] i have done that, followed by a few iterations [22:13] spec is clear now [22:13] yeah, spec is good [22:13] should be easy to xfer that into a set of unit tests [22:13] and then implement [22:13] that is a usability enhancer that would be nice to have in 4.2 [22:14] And who is implementing? [22:14] anyone interested in implementing it? [22:14] sven is listed, but he is focusing on user mapper [22:14] anybody else? [22:14] Are you taking care of sorting as well? [22:15] sorting is out of scope i think [22:15] his is just for linking and jumping [22:15] (sounds like a children game) [22:16] this is just for linking y [22:16] sorting would be implemented in a different place in the code [22:16] Sven you had an old patch? [22:16] i wrote that in 2000 [22:17] i think the implementation is very easy [22:17] that would be unisys, so, unless i suck it onto twiki.org, no [22:17] and localized to one spot (hopefully) [22:17] it could be, the tests are the main thing [22:17] most work is in testing [22:17] to make sure it does not cause other issues [22:17] snapp :) [22:19] i added a note to http://twiki.org/cgi-bin/view/Codev/TopicCaseSensitivity [22:19] asking for help [22:19] next [22:19] # Peter: Create a TWikiBetaTesting topic in Codev, with info for beta testers. [22:20] stil pending :-/ [22:20] needs to be done in time for beta testing phase [22:20] ---+ 2. Review Proposed Features [22:20] If you all refresh the agenda then you can see the list with comments [22:20] kenneth, how do you propose to proceed with the review? [22:21] ah, thanks [22:21] Let me walk through them. [22:21] AddGlobalSettingToAllowNonWikiWords - no disagreement on the feature. One proposal from Peter on the name of the setting. [22:22] http://twiki.org/cgi-bin/view/Codev/AddGlobalSettingToAllowNonWikiWords [22:22] Actually it seems I overlooked that the proposal was altered to the HIDE_NON_WIKI_WORD_WARNINGS that Peter suggested [22:22] i would not really call my comment a concern [22:22] So if this is what all agrees then that is the decision. [22:23] Is the concern about the ability to override the setting? [22:23] (what I understand) [22:24] yes [22:24] i have several sites where i turn off autolinking just for the Publish web [22:24] I have never used the .spec so I do not know if configure settings can be overridden [22:24] e.g. standard autolinking for all but the website content [22:24] Reading the topic more carefully. It was Crawford that proposed a configure setting instead. [22:25] yes [22:25] CDot: comment? [22:25] * CDot reads to topic to recover context; will comment shortly [22:26] others: in the mean time please glance over http://twiki.org/cgi-bin/view/Codev/ProcessAddToHeadAdds [22:26] that one's probly better to defer [22:27] as CDot just proposed what i meant, but we do need time to digest and figure it out [22:27] * SvenDowideit still requires a translator it seems [22:28] ok; I read Peter's comment, and didn;t pursue the point, as I thought he had a valid requirement. [22:28] so, all agree? [22:29] So CDot your concern is gone and you are OK with HIDE_NON_WIKI_WORD_WARNINGS! Then we have a consensus decision [22:29] so the setting should go into TWikiPreferences? [22:29] sorry Sven, I pulled that discussion away from what you proposed (which was correct) in the interests of a simple imple. But you were right at the start. [22:30] crawford: ok on wikiword warning? [22:30] :) [22:30] just shows i need to figure out howto explain why i think in colour [22:31] yes, ok [22:31] I think a couple of ideas are in ProcessAddToHeadAdds [22:32] cleaner formatting of HEAD tags; remove duplicates; setting order; eventually server-side creation of amalgam files [22:32] ArthurClemens: yes, we are converging; but the spec isn;t right yet [22:32] In ProcessAddToHeadAdds - it seems all agree on feature. Can we agree that the proposal is accepted IF/WHEN Arthur and Crawford has agreed with each other? [22:32] no [22:33] when we have a fully functional spec, then its able to be accepted [22:33] OK. Then I put it back in Under Investigation and let the discussion continue. [22:33] right now, if we were to agree&implement it would shoot us in the foot later [22:33] please, yes [22:33] and my feet already hurt [22:34] well, if sven is intersted in the subject we can say that "accepted if agreed by arthur, crawford sven" [22:34] delegate :-) [22:34] giggle [22:34] we have a full year to come to agreement [22:34] anything lese on add to heads? [22:35] its not so much the person thing, rather the correctness thing, but sameo sameo [22:35] ArthurClemens, don't worry, after i have openid done [22:35] i'll spend time unwinding doing these less ouchies [22:35] SearchWithTWikiQueryLanguage is the next. I am not sure we can decide but it is ready for some open discussion. 5 minutes max. [22:35] it is marked as freetown [22:36] sven, in the interest of time, could you glance over the proposal as it evolves and give feedback? [22:36] * SvenDowideit loves CDot for getting there [22:36] at worst, i talk to CDot most days [22:36] but y [22:36] http://twiki.org/cgi-bin/view/Codev/SearchWithTWikiQueryLanguage [22:37] he tql is a very exiting new core feature! [22:37] s/he/the/ [22:37] it would be more exciting if someone would write some decent docs for it [22:37] My view on TQL is - keep it simple. The $something and [?.. syntax is very high on the NerdoMeter. [22:37] I did my usual trick i.e. explaining in greek with a smattering of gaelic [22:37] i feel we are almost there; it is important to consider a solid syntax that can grow over time [22:38] Lavr__: what is $something? [22:38] greek & garlic? yum [22:38] crawford: i can commit to write docs for this feature [22:38] $formfield() was proposed as one proposal [22:38] examples is the biggest thing [22:38] i am retracting from the $something() syntax [22:38] I like he original simple spec CDot made. Normal people can learn to use that. [22:38] read my latest comments [22:39] I'm sorry about the [? syntax, but XPath had the same issue; it's really hard to express any other way [22:39] yes, crawfords proposal before the [?...] syntax was pretty good [22:39] ok, so draft proposals 1&2 are not it anymore? [22:39] it can be done [22:40] example, instead of HistoryForm[?name='Age'].value>2 [22:40] write: formname = 'HistoryForm' AND formfield.Age > 2 [22:40] the [? syntax is *essential* [22:40] or at least, some way of saying the same thing [22:41] right now you can write form.name='HistoryForm' AND Age > 2 [22:41] I do not have a religious preference to the syntax. I just urge you to end up with a simple syntax that you do not have to be a programmer to understand and use. SEARCH is a very important and basic feature in TWiki and it is geeky enough in some aspects already. [22:41] I can;t see what's so bad about that [22:41] CDot, its horrid :/ [22:41] i've seen c# coders get mighty confiused by very simple xpath [22:41] what does the ? do? [22:41] [?...] is almost as powerful as regex, and almost as nerdy as well [22:42] yes, I know. I am wide open to better proposals, but please understand the requirements [22:42] don;t just focus in on simple cases; the general case is *hard* [22:42] the example i just wrote a few minutes ago [22:42] formname = 'HistoryForm' AND formfield.Age > 2 is close [22:42] the problem is that it defines some 'special' names [22:42] Yes. The simple dot syntax is easy to understand and extend later. [22:43] think a few realeases later when we have a wysiwyg editor that has autocomplete with pulldowns [22:43] yes; 'form', 'attachments', 'field' and 'parent' are all special names [22:43] PeterThoeny, I already have that partially implemented in componentedit [22:43] the lack of TOM is what stalled that project [22:44] if you are at [22:44] %SEARCH{ "first.second. [22:44] you get a pulldown of possible choices based on context [22:44] it also needs the Tag work meridith started (in part due to my asking) [22:44] I'm not going to spend ages here trying to explain to you why these proposals, which they are great at face value, don't work. [22:44] that is easy to do with a tree; hard to understand and to implement with [?...] [22:45] I have to let you discover that yourselves - as I did in FWP and DBCache [22:45] because I started there with a "dot syntax" [22:45] yeah, true :/ [22:45] and rapidly ran out of steam with it [22:45] designing a query language is v hard [22:45] crawford: how about this: can we do this project in two phases so that we have more time to work out a solid syntax? [22:46] unfortunately META does *not* describe a simple tree [22:46] which is why i run back to sql - and asking sql for a table def, and then making a dynamic query on that, is v nasty [22:46] and even if it did, more complex TOM structures are *not* simple trees [22:46] 1. simple and limited feature set [22:46] 2. extensive feature set [22:46] point 1 for freetown [22:46] point 2 for georgetown [22:46] if you get 1. wrong, we're totally stuffed for 2 :/ [22:46] PeterThoeny: that's exactly what I have been trying to get across to you [22:46] what I have proposed *is* phase 1 [22:46] what you are proposing - the shortcuts - is phase 2 [22:47] CDot. What you have implemented now is the simple syntax that my mother would be able to use - right? [22:47] if your mother has a degree in EE or CS, yes [22:47] but "simple syntax" is something you learn [22:47] i suggest to remove the [?..] syntax from freetown, e.g. reduce the functionality conciously [22:47] NOOOOO!!!!!! [22:48] so that we have time to create a solid syntax [22:48] you *cannot* do that!!! [22:48] ok, you want me to revert all the query work? I can do that [22:48] I believe it is the "grandmother" test -- not the mother test (that's about the extent of my input :-() [22:48] no, just disable the [?..] stuff [22:49] you *can't*!!! [22:49] or if we agree, redefine the [?..] stuff [22:49] you would *cripple* the query language [22:49] make it *useless* [22:49] I still do not understand the [? syntax and I am an EE. It is very very geeky [22:49] one big Q [22:49] ok, Lavr, let me explain it this way [22:49] you have to understand [22:49] i think the whole proposal started as a need to: [22:49] * form field "Firstname" is "Emma" [22:49] * and form field "Lastname" is "Peel" [22:49] * and form field "City" starts with "San" [22:49] * and form field "Country" is not "Costa Rica" [22:49] * and row "2007" of table named "membership" is "paid" [22:49] e.g. pretty basic [22:49] needs [22:50] Peter, explain how you do a conjunction on a search over attachments [22:50] the example I keep using [22:50] "find all gif files > 1024 bytes" [22:50] CDot: i argue that attachment query can be out of scope of freetown release [22:51] time check: +50 min [22:51] I wish I could make you understand why that is such a bad idea. [22:52] so the ? marks a collection of items? [22:52] it's not just the simple queries you want to make right now; it's *fundamenta* to the concept of TOM [22:52] ArthurClemens: yes! [22:52] i am not suggesting not to add query feature for attachments, i am suggesting to do one step at a time [22:52] Where is the problem about leaving the "complex syntax" there? Nobody is urged to use it. We can have two different places of doccu. [22:52] It represents a search over a collection [22:52] OliverKrueger: THANK YOU! [22:53] ah, thanks, OliverKrueger asked my q [22:53] :-) [22:53] So! The simple task of searching for form field values and ANDing ORing NOTting etc does not require the geek [? syntax ? [22:53] the problem with adding & documenting a feature now is that it is almost impossible to remove later [22:54] Peter, I wish you would hear me on this; we will *never* remove that feature, because it is *critical* to a working TOM [22:54] atm, you have to write a regex to do it... (w/o tql) [22:54] those who need this feature can already use the formqueryplugin [22:54] so I should read it as attachments[where name='purdey.gif'] [22:55] ? [22:55] ArthurClemens: exactly! [22:55] * CDot would be quite happy to extend the syntax to support WHERE as a synonym for ? [22:55] i understand crawford, i am not asking to remove the feature per se, i am suggesting to disable the feature until we have an easy to grok spec [22:55] WHERE looks ok to me [22:56] The spec *is* easy to grok - Arthur just managed it! [22:56] Arthur is a programmer. [22:56] it means you can just read the example code [22:56] yep, WHERE would be a nice to have. [22:56] i understand the current spec, but i think it is too high on the nerdoscale [22:56] one of my client is a lawyer with *no* programming experience - he uses it every day! [22:56] the square brackets are common for collections [22:57] i grok it too :) [22:57] I can live with the geek feature as long as the simple queries for formfield values does not require the syntax. [22:57] seems clear. [22:57] if you don't like geek syntax that is necessary for completeness [22:57] sponsor work on the GUI tools to make them unseen [22:57] crawford, why do we need the query on collection feature now? [22:57] it's more of clarity rather than whether a syntax is geek-only [22:58] * ktwilight likes GUI tools, especially full blown wizards [22:58] * SvenDowideit likes WHERE more than ? too [22:58] but its a tiny irrelevant change to the reality of the feature [22:59] Yes WHERE helps. It makes it more intention revealing. [22:59] we need the syntax now for three reasons (1) because it is how all the shortcuts are implemented (2) because it is needed for TOM (and TOMish plugins are coming real soon) and (3) because without it, some queries over the existing data are impossible [23:00] we need to talk about shortcuts later, lets focus now on the collection [23:00] and not just attachments either; consider a form (this example is from my lawyer client) where there are ten fields; Party1 through Party10 [23:01] he wants to find where a certain person is one of the parties in a case [23:01] so he has to search all 10 Party* fields [23:01] at _which_ party he is right now :-) [23:01] time check: +60min [23:01] ;) [23:02] i have a hard stop at +120 min [23:02] so field[WHERE name~'Party.*' AND Client='nameBeingSearched'] is how he does it [23:02] (last two proposal should not take much time) [23:02] * CDot is typing too fast; risking typos [23:03] the WHERE allows one to search unknown items [23:03] I am saying that I have active clients making queries today; they will be really, really disappointed if the core syntax is emasculated, after they have been used to the power they have [23:03] yes, this is powerful [23:03] and the clients using it are *not* geeks (well, at least one isn;t) [23:03] i do not understand why we need to rush this into freetown though [23:03] those folks who need it can use the fqp [23:03] RUSH IT? After FQP has been there for four YEARS!!! [23:04] I don;t think tht classes as "rushing" ;-) [23:04] Using [WHERE instead of [? actually means a lot to understanding what the feature does when you just see the TQL. [23:04] I'd be happy to add [WHERE ...] [23:04] how can this be done with autocomplete? [23:04] I like it as well [23:04] I think rushing in a good syntax NOW is important because!!! ...... [23:04] or wizard in wysiwyg editor? [23:05] app builder like conponentedit is [23:05] We need it in the field for a year to enable us to more easily change storage technology away from flat files. [23:05] Lavr__: a good point [23:05] yes, that is a good point [23:05] we need tql now [23:06] It is important that we - in the 4.2 release notes - describe the new TQL feature and announce that in future searching for exact patterns in meta may not work in new storage implementations. [23:07] the question is, can we afford to redefine the syntax in the future when we realize that it does not work well for content access syntax or wizzards, or that people have a hard time to grok it [23:07] i just had a thought [23:07] yes, PeterThoeny i think we can [23:07] for sure [23:08] because you can transform any simple syntax into a more powerful one [23:08] the reason why we couldn't before is regex's don't carry enough info in them [23:08] but the reverse is not true [23:08] you know my opinin on deprecating syntax [23:08] i always wanted to transform and syntax check tml [23:08] but the use of regex's made that too painful to do where it mattered most [23:08] whereas this kind of syntax is less of a problem for that [23:08] steve, as a customer advocate, what do you think? [23:09] agreed on regex, this is very nerdy, and also storage implementation dependent [23:10] worse than nerdy, there was no way to know that the regex you make, and the one i do, actually are looking for the same thing [23:10] the reason I added regexes was simply because I have a lot of experience of SQL, and LIKE drives me nuts [23:11] they are basically lossy, lowest level things [23:11] howwver I *can* be persuaded to drop general purpose REs in favour of a LIKE-like construct [23:11] CDot just to clarify. (The proposal topic is not as clear as it was in the beginning now). %SEARCH{"Firstname='Emma' AND Lastname='Peel' AND City~'San .*' AND Country!='Costa Rica'" .. is that still the most simple syntax? [23:11] yes [23:12] Question: If Twiki is moving to a backend store (eg., MySQL) wouldn't making the tql more SQL-like be somewhat advntageous training-wise"? [23:12] OK. That is for my mother then. And then I will buy the [WHERE. [23:12] its (possibly) useful to still be able to use regex's [23:12] SteveRJones: possibly; that's an argument for LIKE [23:12] as then we can xform worstcase existing searches [23:12] Not that I LIKE the syntax --- it's just in such wide use. [23:12] what i did in DatabaseStore was to fall back (as a last resourt) to regex [23:12] Ability to use Regex for the values is still nice. The problem with current SEARCH is that you need to regex search for patterns related to how the meta data is stored. [23:13] as there will always be old users [23:13] SteveRJones: SQL is actually a lot more powerful than we can support [23:13] Lavr__: the problem is the recognintion that DB backends don't usually have RE searches [23:13] at best, they have LIKE [23:14] Yes. I know. And if you implement regex as "look at value of all records" then we may as well stay with the flat files. [23:14] right [23:14] CDot, Lavr__ we have to do the dumb way anyway [23:15] though you myay be able to transform the RE into a 2 pass match [23:15] do a LIKE first, and then an RE on the results of that [23:15] yep [23:15] Yes. DOing the LIKE on the static part of the regex. [23:15] but yes, RE's are potential pain for backends :-( [23:15] good discussions, in interest of time let's continue this on the proposal topic [23:15] the detail is that the last resourt create a set of v1.0 topics and run grep will still have to be there [23:16] and thats _not_ a bad thing [23:16] yes, please, feedback and *especially* experimental results are welcome [23:16] we need to decide now what syntax is in freetown [23:16] anyone that uses it will be well aware of the slowdown :) [23:16] see AllOutStandingItemsQuery for my example [23:16] what, it's slower? [23:16] yes, it will be slower than text searching at first [23:16] for sure; it has to load all topics [23:16] Yes. It was a good discussion. CDot. Can you wrap up the summary of the decision stating both the simple and the [WHERE? Then we may reach consensus on the topic in a few days. [23:17] Lavr__: sure [23:17] NEXT ITEM. [23:17] ArthurClemens: it will soon be *much* faster [23:17] "watch this space" [23:17] no, one more item on tql [23:17] shortcut [23:17] i suggest to remove the shortcut to point to formfields [23:18] e.g. always start with a qualifier [23:18] Hmm. That is the part I like because it lower the NerdoMeter score for the simple and most used cases. [23:18] formname.something, fowrmfield.something [23:18] erm, that's gets very noisy [23:18] queries tend to get lengthy... [23:19] a shortcut in a query language is ambiguous [23:19] yes, there is that risk [23:19] I am sure my mother will easier learn firstname="Kenneth" AND Lastname="Lavrsen" than formfield.firstname="Kenneth" AND formfield.Lastname="Lavrsen" [23:19] a formfield are just another item in the tom [23:20] how about $firstname='Kenneth'? Is that also evil? [23:20] sorry, yes, of course it is [23:20] I find that is mixes up the syntax used in format strings. [23:21] but if you have formfields[WHERE firstname="Kenneth" AND Lastname="Lavrsen"] that is not ambiguous [23:21] ArthurClemens: right [23:21] your mother will like that there arel items called formfields, and that you can query them by typing: formfield.firstname = "Kenneth" [23:22] she would be delighted [23:22] right now "firstname='Kenneth'" is equivalent to "field[WHERE firstname='Kenneth']" [23:22] oops, I meant "field[WHERE firstname='Kenneth'].value" [23:23] Scream [23:23] that's why I added the shortcut :-/ [23:23] formfield.firstname [23:23] or [23:23] formfield.firstname.value [23:23] is much easier to understand [23:24] and it avoid the ambiguity of just "firstname" [23:24] no, it doesn't, I'm afraid [23:24] what would "firstname" be confused with? [23:24] it assumes that all arrays are indexeable by a field called "name" [23:24] so: [23:24] formfield.firstname = "kenneth" [23:24] vs. [23:24]