Session Start: Mon Nov 05 22:04:24 2007 Session Ident: #twiki_release [22:04] * Now talking in #twiki_release [22:04] * Topic is 'http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/ReleaseBlockers' [22:04] * Set by CDot on Mon Oct 08 10:19:21 [22:04] Good evening Gilmar. Great job with standalone! [22:04] no, Peter mentioned GMT 21:00 [22:05] According to http://twiki.org/cgi-bin/xtra/tzdate it should be now [22:05] Good evening [22:05] thanks! :) [22:06] Hi Lavr_. [22:07] Are Peter, CDot and Sven going to attend? [22:08] Peter wrote himself as attending [22:08] Without Sven and CDot it will be a bit pointless because they wrote most of the code of 4.2 [22:08] Yes I agree [22:08] So they need to explain what was intended [22:08] But it gives a chance to maybe discuss some of the less "sexy" bugs [22:10] We now have a beta 3 but not even I can install it at work because some of my very popular plugins no longer works so we need people to start updating plugins now [22:10] It is especially those that uses templates that are broken. [22:10] Sorry, but I have no idea what may have broken them [22:11] ...and *why*. [22:11] It is the updates to the TWiki templates that are not compatible with 4.1.X [22:11] So any plugin that has templates that build on the TWiki default templates or pattern templates break [22:12] Ah - pattern. Love it or leave it :-) [22:12] * ktwilight has joined #twiki_release [22:12] oh oh, my bad :) [22:12] hi [22:12] Those plugins I am thinking about also stopped working on classic skin [22:13] * LarsEik has joined #twiki_release [22:13] HistoryPlugin was broken last time I tested. So is CompareRevisionsAddon/Plugin [22:14] Lucky me did not yet install CompareRevisions, though I often wanted to. [22:14] It is a plugin with a function which in my oppinion should replace our current diff in core. [22:14] * ktwilight didn't know it exists [22:15] i would agree to that if it does more than what diff can. [22:15] diff is too technically crafted IMHO [22:15] burp :) [22:15] nice [22:15] Example of how it works. http://www.lavrsen.dk/twiki/bin/compare/Motion/FrequentlyAskedQuestions?rev1=54;rev2=53 [22:15] if you don't like how pattern skin changes... use MoveableTypeSkin [22:15] I don't think it is really difficult to update the templates - it just needs to be done [22:16] it'll not change [22:16] there is no question other skins may be more attractive [22:16] diff on natskin isn't that great either... [22:16] (I _like_ the changes to pattern, but those are 2 different focuses) [22:16] but if it is broken on default skin it is of no use either [22:16] cool, i like compare, it's more UE friendly [22:17] If anyone really works on it [22:17] on what? [22:17] can they please just push the compare func into diff?? [22:17] I wrote diff to be extendable [22:17] but no-one ever did [22:17] perhaps noone knows [22:17] There is too much refactoring refactoring refactoring on the templates. We need the essential part of that frozen as an API. [22:17] twas in cairo [22:17] [off-topic] Is there some statistic about pugins' popularity? [22:18] GilmarSantosJr, good q - i don't really think so [22:18] would be nice if there's one [22:18] best we have is dwnload stats [22:18] which is more a measure of the plugin's teaser blurb [22:19] up early eh SvenDowideit_? [22:19] yeah :) [22:19] 8am! [22:19] the refactoring was done to create compatibility again [22:19] o, i would imagine it's 5-6am. [22:19] thats like 2 hours before 'normal'!! [22:19] haha [22:19] we got daylightsavings [22:19] damn time changes. [22:19] because I made some incompatible changes earlier [22:20] (noone mentioned) [22:20] ArthurClemens, yeah, broke those people that fixed their plugins to meet the first change :) [22:20] damned if you do.... etc [22:20] * PeterThoeny has joined #twiki_release [22:21] some plugins are out of my reach [22:21] y, and some are just horrible to work on [22:21] QuickMenu or something [22:21] sorry, i am late, i had a router/firewall issue [22:21] do we have a list of 'em? [22:21] I've taken to using perltidy on code [22:21] hiya peter [22:21] and i'm working on using perlcritic, though thats pretty depressing atm [22:22] whats that? [22:22] hi arthur, gilmar, koen, harald, kwan, lars, kenneth, sven [22:22] hi peter [22:22] good turnout! [22:22] hi peter [22:22] perlcritic critisises your code [22:22] based on current known perl best practice [22:23] i wonder [22:23] cool [22:23] has anyone _here_ tried one of the installers for beta2 or 3? [22:23] where are we in the meeting? [22:23] I wish people would stop being buried in all sorts of unit tests and other tools and just start plain using TWiki from a browser so we can find the remaining bugs. [22:23] na, freeform chaos [22:23] Lavr_: did you have the list of broken plugins as Bug items? [22:24] Lavr_, how do you think i've been finding these rest bugs? [22:24] History and CompareRevisions are the ones I tested and that I found broken. I stopped there. [22:24] I'm busy writing new plugins & cntribs to test our api's [22:24] * ktwilight thumbs up [22:24] in any case all plugins that have a *.pattern.tmpl customization [22:25] mind you [22:25] Peter we have not started. I brought up the issue that people cannot install betas because the most popular plugins are broken in 4.2 [22:25] I can report that we've had about 2,000 downloads of the installers [22:25] so I would suggest that this twiki release is probably the most tested ever [22:26] do we have bug reports from those 2000? [22:26] Not when you look at Bugs web. We are only max 10 that reports bugs at the moment. [22:26] I doubt so [22:26] well, I feel we have had bug reports from new people y [22:26] and I've had a couple via email [22:27] mmmm, there's a report we need [22:27] we need to find a way for people to file more bug reports in our busg web (email inboxes are not good to track bugs) [22:27] who created bugs [22:28] I xfer them to the bugs system [22:28] good [22:28] I don't use my inbox for it, cos then i'd have to fix it too [22:28] shall we start with the formal meeting? [22:28] fine [22:28] who is facilitating, who is taking minutes? [22:28] Looking at bugs - we have very few non-working on urgents at the moment. [22:28] http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/WebStatistics [22:29] SvenDowideit_: are plugins download statistic available somewhere? It would be nice to have some "popularity rank" in order to make tests... [22:29] Lavr_, ? parse error [22:29] i can facilitate [22:29] who is on minutes? [22:29] I start minutes [22:29] thanks [22:29] ---+ Action Item Review [22:29] GilmarSantosJr, y, somwhere :) [22:29] i'll add you to the list [22:29] # CarloSchulz: Open a new bug item about the installation docs [22:29] status? [22:30] We had a long debug session after the previous meeting [22:30] on install docs? [22:30] but AFAIK no bug report, nor a report for TWiki on hostmonster's yet [22:31] i miss something, what is missing in install docs? [22:31] The session (on #twiki) was intended from my side to understand why Carlo had difficulties with installation [22:32] I think there's still too little guidance for typical hoster setups [22:32] so purpose is to see what can be improved in install docs? [22:32] That was the idea, yes [22:32] can someone ping carlo to see if he can help on docs? [22:32] GilmarSantosJr, I've added your user to have perms for http://twiki.org/awstats/awstats.pl [22:32] Carlo seemed to be confused which directories need to be exposed to the web server, and which don't [22:33] SvenDowideit_: Thank you! :) [22:33] And fixing that after everything is in public directories is a mess because configure, once it has guessed its values, has to be corrected in a tedious line-per-line process [22:34] ok, problem is understood [22:34] who is following up? [22:34] carlo and? [22:35] I've promised to help Carlo with what I think is "best practice" [22:35] halard, would you have time to see if the install docs can be tweaked in this regards? [22:35] ok [22:35] But I have too little experience with hosters and what their setups are [22:35] lets move to next item [22:35] ---+ 1. Review Urgent Bugs [22:36] http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/ReleaseBlockers [22:36] HaraldJoerg, talk to me a little more about it later [22:36] kenneth, do you want to pick some? [22:36] Yes. [22:36] http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/Item4570 [22:36] Sven and I had a good discussion about this. I added a link to the discussion on the bug item about 15 minutes ago. [22:36] I listed our options. [22:37] The ones I can remember. Sven you may want to add a few. [22:37] Steffen had a strong feeling on this one - sad he is not here [22:37] "We can accept the risk" [22:37] My view is to go for the at least the docu solution. [22:37] its not us that accepts the risk [22:37] we're forcing it on others :( [22:37] afaik, cfg{DefaultUrlHost} is only used by mailer, e.g. non-http scripts [22:38] Yes. I am putting options. [22:38] http-scripts should take the domain from the url [22:38] No. If you set cfg{DefaultUrlHost} to www.lavrsen.dk then TWiki will not work with lavrsen.dk [22:38] \this used to work just fine [22:38] PeterThoeny, thats what is the problem (spoofing wise) [22:38] You can view but you cannot edit/save cycle because that generates internal redirect cycles. [22:38] Confirmed. That's the same if you want to mix https and http [22:39] we had to remove it for the redirect cve [22:39] but we should have removed it for all operation (for similar security reasons) [22:40] basically, the way apache works by default is a phishing risk to third parties [22:40] I still have not seen the attack vector anyone can abuse if we let the redirect accept $ENV{'REQUEST_URI'} for redirect [22:40] but its normally mitigated by the fact it serves static like content [22:40] o, didn't see that. [22:40] because twiki is totally dynamic - imaging a nicely crafted VIEW_TEMPLATE, and a set of topics [22:41] $ENV{'REQUEST_URI'} is set by Apache to the domain the client used to access. This domain is EASY to spoof. [22:41] And when I say easy I mean really easy [22:41] excatly [22:41] why does twiki need it whilst others don't? [22:41] ktwilight, ? [22:41] assuming that other webapps don't. and if they do, what is their workaround? [22:41] its always been there as a conveinience [22:42] other webapps are either more secure, or have similar gotchas [22:42] ah right. [22:42] but few allow their entire look and feel to be modified [22:42] which is what makes twiki more difficult than most [22:43] we have to look harder at some security issues, because we open up others [22:43] being able to add literal html to a topic, and have topics as view_templates [22:43] means we _must_ be very sensitive about tiny phishing vectors [22:43] whereas most webapps can ignore them [22:43] because they can't be made to look like other sites so easily [22:44] The CURRENT situation is that people can setup TWiki to both accept www.lavrsen.dk and lavrsen.dk and maybe a couple more by putting the most common in {DefaultUrlHost} and a commalist of the others in {PermittedRedirectHostUrls} [22:44] The {PermittedRedirectHostUrls} is a hidden EXPERT setting. [22:44] Lavr_, i think you've not saying it right [22:44] far away in configure from {DefaultUrlHost} [22:45] The CURRENT situation is that _redirects_ will only work for {DefaultUrlHost} and {PermittedRedirectHostUrls} [22:45] So the SAFE but user friendlier fix is to maintain current function but document the facts a little better and UN-expert {PermittedRedirectHostUrls} [22:45] but twiki _still_ (this is a security bad thing) will allow viewing on REQUEST_URI [22:45] and will re-use that uri for html linking [22:46] i vote yes to your docco suggestion though [22:46] * ktwilight didn't even know about PermittedRedirectHostUrls [22:46] y, i added it recently [22:46] ah [22:46] Users will raise support requests in great number because of this issue. [22:46] * ktwilight thought it's there in 4.1.2 [22:46] na [22:47] In 4.1.2 you can access and edit using any domain name. [22:47] Lavr: I don't think this will raise too many support questions. [22:47] yes. It is extremely common to accept both with and without leading www., [22:47] for the good of twiki, support questions is really secondary IMO. [22:47] I think it was there for a long time. I had problems with stand alone (running on port 3000) and fixed adding it to PermittedRedirectHostUrl [22:47] we can always lower the number of support questions via good doco and clear indications/signs [22:47] The best workaround is to canonicalize hosts with Apache virtual hosts [22:47] HaraldJoerg, yup. [22:47] though, how 'bout non-apache webservers? [22:48] HaraldJoerg, unless of course the user is using non-apache :/ [22:48] snap :) [22:48] reminds me [22:48] Every web server should offer that sort of functionality [22:48] Unless you setup Apache to rewriting the URL the virtual host will not resolve the issue. www.lavrsen.dk will still be different from lavrsen.dk [22:48] anyone here up to looking at the IIS cgi driver that was released a few months ago?? [22:49] Lavr_: You need to redirect the client, yes. [22:49] HaraldJoerg, y, but unless its been abstracted by someone, someone has to implement and test it :( [22:50] Implement *what*? [22:50] knowing about the virtual host settings in the web server [22:50] or can you ask the CGI module for that? [22:50] I agree with Kenneth's proposal: aditional doc and UN-expert {PermittedRedirectHostUrl} [22:51] time check: +50 min [22:51] un-experting sounds like a sensible thing [22:51] I think it's safe and easier to configure then apache rewriting rules (and more portable - doesn't tied to apache) [22:52] You don't need rewriting rules! [22:52] And if you want to, you could do the redirect from within TWiki. [22:52] And move {PermittedRedirectHostUrls} up in General Path Settings just below the {DefaultUrlHost} [22:53] It should be a clean change in the spec file. [22:53] The only use case I can see for having two host names in parallel is if you have different networks accessing the same server [22:53] If we can agree on this then I will implement that for 4.2 [22:53] HaraldJoerg, lots of people seem to use it [22:53] and there are other reasons [22:54] Featurism. [22:54] I use t42p, 10.10.10.180, t42p.home.org.au and a few other names [22:54] and there are others [22:54] for maintenance it is useful to access server by ip address (before dns is set), so multiple domains should be supported [22:54] it is useful [22:54] but it is also a phishing risk to toher sites [22:54] and that means it needs to be made less automatic [22:55] (or, we remove inline html in topics) [22:55] it looks like we have consensus on un-experting? [22:55] y [22:55] ok, so lets move to the next one [22:55] y (and moving it near to DefaultUrlHost) [22:55] Sven I still do not get how you can redirect to another site by allowing redirect to REQUEST_URI. THAT should be documented how someone would exploit that [22:56] thats not the exploit i'm suggesting! [22:56] But if we can agree on my solution then we are both reasonably user friendly and safe [22:56] your solution is part of it, and i agree needed [22:57] the phishing can be done using same domain, different hostname too.. [22:57] kenneth, sven, could you take the details offline? [22:57] yup [22:57] in interest of time, lets move to the next one [22:58] i've just updated http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/WebStatistics [22:58] OK. No objections. I will implement my proposal. We can always leter extend with the "unsafer" REQUEST_URI in a 4.2.1 [22:58] IF we agree to then [22:58] seems to me there are more than 10 people reporting bugs [22:58] Lavr_, I agree [22:59] OK next item.. [22:59] http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/Item4918 [22:59] checkpoint-save ignores EDIT_TEMPATE [23:00] Michael has proposed a fix so I guess we just need some expert evaluation of it [23:01] Has anyone looked at it? [23:01] tbh no - micha knows more about it than i do [23:02] If someone see him the next days on IRC maybe ask why he does not checkin his fix. He must have some doubts. [23:02] i lack the context to say if/how this affects the actions [23:03] Next item. [23:03] * Bugs:Item4898 - tmp and working dir security needs work (docco and code [23:03] time check: +60 min [23:03] http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/Item4898 [23:03] how much longer shall we go on? [23:03] Sven this is yours [23:03] 30 more min? [23:03] I have round 5 more and they will not take as long as the first [23:03] Item4898: tmp and working dir security needs work (docco and code) [23:04] basically, I had to fix a cve for debian [23:04] If you want to go through all release blockers I'd call 30 min too optimistic [23:04] in which a number of issues wrt the tmp dir and sessions were brought up [23:04] Most already have (Being worked on) [23:04] we only cover some that need attention [23:05] sessions files for eg should not be made for non-cgi requests [23:06] hmm, some hosting providers runs scripts on regular basis to change 777 to 755 [23:06] ok, updated http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/WebStatistics again, this time with top 30 saves [23:06] thats not enough [23:07] as there are risks (that were explained to me) due to the ability of others to create files [23:07] what i am saying, twiki cannot rely on a 777 [23:07] thta then get picked up by twiki as though it created them [23:07] and we should try to avoid it [23:07] its not [23:07] It's more complex than that. Plain Linux doesn't allow chown unless you're root, so permissions are often not set "secure" [23:08] twiki is allowing it, but it is also allowing anyone to give it a file that look like one of its own [23:08] We should set the tmp access right to the default we use for most other things and then let experts that run non CGI scripts setup what they need for that [23:08] TWiki has no "role model" for installer, operator, web server so everything is "it used to work that way" [23:09] i think we should have relatively tight permissions for working dir [23:09] We actually made a big discussion about access rights for 4.0 and our current approach was a comprimize that allowed shared host customers (that are often the least experts) to install TWIki and get it to work [23:09] experts know what to do if they need non-webserver users to interact with content [23:10] When we decided to have world access to files it was because shared host installers often needed it. If you change to 770 then they have trouble again. [23:10] one issue seems to be that we often make such a decision [23:11] even though a security person would tell us we're being foolish [23:11] there are too many little places where we open ourselves up for conveinince [23:11] In a deb package or an rpm or in an installer we can make it tighter because shared hosts cannot install these anyway. They upload via ftp. [23:11] all of which add up in places that are not important to normal web apps [23:12] hosters also use the rpm's [23:12] Then it is an expert that installs it with root access who can change access rights if they are default too tight. [23:12] as you can relocate the base install dir, and then have a oneclick installer for users [23:13] basically [23:13] I'd like it to be secure by default [23:13] and experts can loosen it [23:13] right now, this is _not_ the case [23:13] mostly because none of us are very good at security [23:13] I think it would help a lot if there was documentation which directories need to be accessed by which users, so that experts have a chance to align it with their policy [23:13] because we liek to _use_ twiki [23:14] good point harald [23:14] security suggests we should re-write the portions that atm require other user write perms [23:14] which the reason for my suggestion at the bottom of that bug [23:14] is TWiki/SettingFileAccessRightsLinuxUnix suffice to say the least? [23:15] ktwilight, no [23:15] Peter you were the one that pushed through that we have world read access. Is this still your position? [23:15] its the best we can do atm [23:15] ok, what are the shortcomings of relying on SettingsFile* [23:15] the code [23:15] did i? [23:16] TWiki/SettingFileAccessRightsLinuxUnix sets world read access. But that can be modified so you can choose like I did on ApacheConfigGenerator. [23:16] ktwilight: The first shortcoming is that it starts with "chown -R apache:apache twiki" [23:16] If you aren't root, you can [23:17] can't do that [23:17] So it fails from the beginning for most hosted sites [23:17] Exactly. The shared host customer cannot do things as root. His files have himself as owner when he ftp them up [23:17] kenneth, yes, i did: http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/Item3280 [23:18] hm, someone quick! start a hosting company! [23:18] :) [23:19] So the default directory for working should be 755. Like it is for data today. [23:19] yeah, i was going to suggest we use the twiki.net domain to set up a twiki hosting business [23:19] i guess the appropriate question is, what code changes are needed to overcome such inconvenience? [23:19] where all proceeds goto the twiki project [23:19] ktwilight, code changes [23:19] so that we only create files where really needed [23:19] I will look at TWiki/SettingFileAccessRightsLinuxUnix so it gets a parameter that turns off world. [23:20] SvenDowideit_, ?? are you saying now twiki is able to create files wherever? [23:20] what is the reasdon to trun off world read? [23:20] atm, we make session files for all requests, even though they are only useful for cgi [23:20] SvenDowideit_: that sounds like a solid business model, making sure you can't host it on a generic hosting account so that they _have_ to come to you :) [23:20] gmc, haha [23:20] good point [23:20] crippleware open source [23:20] woohoo! [23:20] no-one can modify it to fix it [23:21] Peter. On a server that ONLY serves TWiki and has no users - it makes no difference in practical. Hackaer have the rights as apache anyway. [23:21] Lavr_, incorrect [23:21] the cve was based on local user [23:22] I talk about the case with no local users. [23:22] _we_ have to consider all use cases for security [23:22] how about leaving as 755 and 644 and document to tighten it for sahred hosts to 750 and 640? [23:22] not dismiss some [23:22] shared hosts are not the target users [23:22] they are the _users_ [23:22] No. it is the shared hosts that need the 755 and 644 and only in some cases. [23:22] so for security, we can't be choosy [23:23] we have to consider all use cases for security issues [23:23] Which shows that shared use is insecure by definition. [23:23] no [23:23] perms is only a small part of the equation [23:23] bad coding like that brought up by that bug [23:23] I remember I could log into Sourceforge on my Motion project and without any problem fish our the TWiki .htpasswd file. [23:23] (s/bad/insecurre/ [23:24] is much more than just die to perms [23:24] s/die/due/ [23:24] i rember tons of installation questions and issues in the support web because of the no-world read flag [23:24] yes [23:24] being secure is always more painful [23:24] What is the conclusion on Item4898. We are for sure not changing back to no read access. [23:24] pretty irrelevant compared to the impact if someone exploits it in a major way [23:25] considering most people here seem to be advocating we continue to be insecure [23:25] But not 777 either. [23:25] i now recal why i don't raise security issues much [23:25] on Item4898, can we tighten 777 to 775 or 755? [23:25] that is not the core issue? [23:26] unfortuanatly, i do have to go [23:26] It needs to be 755. That is our normal default. [23:26] HaraldJoerg, what's your opinion on tackling this issue? [23:26] to re-iterate [23:26] the file and dir permissions ARE NOT the problem in this bug [23:26] they are only a part of it [23:27] bbiab [23:27] I will continue to next [23:28] http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/Item4771 [23:28] TWikiUserMapping fails with Corporate SSO [23:29] that is a good one, sso is used more and more [23:29] Harald and Sven were the players on that one. It is getting pretty old and seems to move nowhere [23:30] For info. It actually works with SSO. It was fixed. There is some minor outstanding issue. [23:30] This is the reason why I can't use 4.2 in my corporate TWiki [23:31] I have lost track of the issue. It the remaining that you cannot register if authenticated? [23:31] Yep [23:31] That is a release blocker. We need this fixed ASAP. [23:32] And it is only when you authenticate the view script by the way which is NOT The recommended normal usecase. [23:32] ah, that needs to be fixed. i have a number of installs where all scripts are under auth, e.g. where an already registered user regsiters new users [23:32] Normally you authenticate viewauth and not view [23:32] how about creating a new bug report to track this new/related issue? [23:32] and close the sso one? [23:33] time check: +90 min [23:33] What does that change? [23:33]