create new tag
, view all tags
It's now over 2 months since the LocationLocationLocation patch was first uploaded, and the test environment has been running for several weeks. There have been a few reported problems, all of which have been fixed. There is no other test feedback, positive or negative. But there is still no indication that the patch will ever be merged to MAIN. In the interim, work on the core in the DEVELOP branch is "blocked" while we wait for feedback. Any feedback.

How can we move ahead from here?

SvenDowideit has suggested opening up DEVELOP to check-ins that are not related to the L^3 patch.

There are two significant pieces of work waiting for merge; RegisterCgiScriptRewrite and ReleaseLocksOnSave. Of these, ReleaseLocksOnSave touches a large amount of the system. Merging this would make it much harder to merge the current DEVELOP branch to MAIN, but it would allow developers to get on with things (like fixing the security issues!)

If we open DEVELOP to checkins, the core team will have to sort out how it is going to handle mergeing to MAIN.

-- CrawfordCurrie - 19 Nov 2004

Don't open it, not yet. Let's wait until the security issue is resolved.

CoreTeam, can we please merge DEVELOP and trunk and release a beta soon? If the merge result in a unstable relese, then we can either go forward and fixit or rollback the merge. There isn't much to fear.

On the other hand, there are few of us with patches ready just waiting for prime-time in DEVELOP (CrawfordCurrie just mentioned 2)

If DEVELOP is not merged from time to time into trunk (let's say, monthly or so if stable) we will have in effect two versions of TWiki (trunk and DEVELOP) and developers would not know against which one to work: According to the development model, we should work against DEVELOP, but if it never gets merged or get merged like twice a year then there is more "productive" to generate patches against trunk, specially for quick fixes on small issues, as they become more likely to make it into the core. This is a step backwards in the way twiki is being developed.

The main question is: how can we guarantee the stability of DEVELOP?. Well:

  1. We limit the access to DEVELOP to contributors that have already proven that they want to contribute.
  2. We have many eyes on the code that can change anything that must be changed.
  3. Tests, tests and more tests.This is already being implemented, and I hope that the TWikiTestInfrastructure will grow in time. So it will be easy to judge the stability of DEVELOP to decide if a merge can be done.

So, CoreTeam, what do you say?

-- RafaelAlvarez - 19 Nov 2004

I'd say, don't make DEVELOP into a dead facsimily of MAIN with everyone waiting on each other. so long as you follow the process (provide automated unit tests & docco) for each of your changes to DEVELOP, life should be good.

Don't wait for MAIN, we're often busy at the wrong times. if you don't have / don't want DEVELOP access, develop patches against whichever version you need, but bear in mind that the changes are more likely to be merged into DEVELOP.

DEVELOP is an experiment to see if an open, unrestricted process helps us get over the intermitent progress that we have suffered from for years.

there are a number of things in progress for the security issue - right now, the first to commit them should win (assuming you remmebered your tests and docco).

I do not want to see stability used as the excuse to delay commits into DEVELOP - if you really need stable use MAINLINE, and consider joining the Core team. That said, if its not stable, write a bloody unit test already!

I'm sorry - I can't see the confusion about what version to develop against - if you are developing anything, develop against DEVELOP, and use DEVELOP - if DEVELOP turns out to be stable, and better than MAIN, it will probably become the new tree that we release from. (don't quote me on that smile )

-- SvenDowideit - 20 Nov 2004

I can sense a level of frustration. Actually the same on my side. In LocationLocationLocation I stated several times that it is in everyone's interest that the MainBranch and DevelopBranch do not diverge too much. It is also agreed on that issues in DEVELOP should be resolved before we accept it for merge.

Please allow me to state the issue one more time: The TWikiMission clearly states to "Protect corporate investment (topic contents) from data corruption and incompatible changes - don't require the use of upgrade scripts". I am very concerned that the rendering order change will break some TWikiApplications. Quite a while ago I invited Crawford to install DEVELOP at my workplace and that I will help test it on the mission critical applications we have. This has not happened yet.

The new process is not working and needs to be revisited if we do not find a solution real soon. That means: Verify what breaks and decide if acceptable, or revert the change. Simple as that. Sorry, that might come across tough, but we can't ignore our 100K+ customers.

-- PeterThoeny - 20 Nov 2004

Peter, I refer you to my mail of 4th November describing the DEVELOP installation at your workplace, which has been available since that time.

  • Crawford, I do not recall receiving the e-mail. I discovered the installation on the server myself and tried it out but it did not work (failed to follow links and missing Plugins). -- PTh

-- CrawfordCurrie - 20 Nov 2004

Given that this is a frustration-pouring topic, here I go.

The current development process, in fact, is broken. But is as broken as the old development process, and suffers for the same malady: Changes are not integrated into core fast enough.

No, let me reparaphase that: Changes that don't come directly from the CoreTeam are not integrated fast enough.

More so, it's not a help either that the CoreTeam DON'T follow the current development process and are checking in directly into trunk (leaving the responsability to merge them into DEVELOP to whoever contributor notice and care, like CrawfordCurrie).

Each time a step forward is proposed, the same phrase is used (it has become a kind of mantra):

The TWikiMission clearly states to "Protect corporate investment (topic contents) from data corruption and incompatible changes - don't require the use of upgrade scripts".

And that's ok, I agree with the TWikiMission and is very important to have a set of goals that drive the development. But the key word is "Drive". Currently, is "Slowing".

Let put the cards on the table, and verify if the LocationLocationLocation patch goes against the TWikiMission:

  1. Enhance TWiki evolutionarily to fit the mission. Ok, DEVELOP is evolutionary
  2. Easy maintenance is key. From the user point of view, DEVELOP is as maintenable as MAIN
  3. KISS (keep it simple) design is preferred over complicated designs. DEVELOP simplifies the design
  4. Design for a good "out of box experience"; advanced features are OK as long as they do not distract. DEVELOP is faster than MAIN, so it has a better "out of the box" experience
  5. Keep CreepingFeaturitis to a minimum. LocationLocationLocation don't add new functionality. DEVELOP has no new "far-fetched" features
  6. Low requirements on client side and server side as defined in RequiredEnvironmentForTWiki. No new requiremets, so far
  7. Keep the quality as high as it is, e.g. few or no bugs for production releases and deployable in many environments. There have been no bugs reports in a long time, and the testcases are increasing in number to guarantee that
  8. Give broad corporate appeal, without sacrificing above guidelines. DEVELOP is as appealing as MAIN to corporation... well, at least it will be as soon as the changes in PatternSkin are merged into DEVELOP)

Ok. Thats 8/8, and only leave us:

Protect corporate investment (topic contents) from data corruption and incompatible changes - don't require the use of upgrade scripts (e.g. category to forms upgrade happened automatically)

There are 3 statements in one:

  1. Protect corporate investment (topic contents) from data corruption DEVELOP don't corrupt the existing data
  2. Protect corporate investment (topic contents) from incompatible changes Is stated clearly in the developer's documentation that all access to twiki functions whould be made via FuncDotPm, or they risk to break in future versions of Twiki. That means, if a TWikiApplication and Plugins breaks because the calls stuff directly from the other TWiki modules is NOT our problem. Is THEIRS. Of course, the sensible thing to do is to help them upgrade is document all the moved, removed and changed subs and variables along their current substitute via FuncDotPm or the new location. AND, follow the proposal to put FuncDotPm along the plugins so people can write workarounds and submit them back (TODO: Find the topic where it was proposed).
  3. Don't require upgrade scripts DEVELOP don't require upgrade scripts

The only valid concern is that the new rendering order can break some TWikiApplications. So lets see the scenarions:

  1. the TWikiApplication depends on the rendering order so parameters are rendered before the main tag, then they'll work exactly the same in DEVELOP (DEVELOP rendering process does just that).
  2. If the TWikiApplication depends on the Plugins ordering, then it will work exactly the same as plugins are not changed.
  3. If the TWikiApplication (NOT a Plugin) depends that some tags are rendered before others (and here we're going to the very outer edges of far-fetchisism) then perhaps it will break.

So, the only thing that is stopping the merge is the third one, but the argument don't hold without tests. So, let's write some tests to prove that DEVELOP break some TWikiApplication and let's make them pass.

Moving forward is the best way to advance.

-- RafaelAlvarez - 20 Nov 2004

Rafael, I could not agree with you more.

Peter, can you share a test with us that fails? We can then set about rectifying. This will make sure we are not dealing with speculation, that we do have direction and we will get some completion.

-- MartinCleaver - 20 Nov 2004

I can test this as soon as I have a working DEVELOP at work.

-- PeterThoeny - 20 Nov 2004

i'd like to state that DEVELOP is not encumbered by the limitation of MAIN, and so can quite happily 'break' legacy applications so long as there is a good reason. L^3 and the deterministic tag rendering order (and therefor testable) is a good reason. I did not put the large amount of work into DEVELOP (and will not continue that work) if it is stifled in the same way as MAIN is, even if the reasons are reasonable.

My entire reason to create DEVELOP was to give the TWikiComunity the oportinuty to try to work differently from the MAIN. and this includes being able to make their own goals and rules.

If this is not acceptable, then we may need to make another branch for that purpose, and I will work on helping the TWikiCommunity there.

I think its pretty obvious that I do not think that Peter's reasons are acceptable for an open source project (at least not for this long).making everyone suffer for the backwards compatibility of some undocumented (where are the unit tests...) issues is rude and disrespectful, and for me, totally unacceptable.

It is acceptable to me to have a TWikiCommunity driven branch (DEVELOP) that is used as a staging ground leading the way that MAIN and TWiki go.

so, to state it differently, there should be no thought of reveresing anything out of DEVELOP for legacy reasons, unless there are unit tests that fail, and they are seen as totally irreplaceable buy the community. The same applies in reverse, not code should be allowed in unless there are unit tests to support it.

I am not WindRiver support.

-- SvenDowideit - 21 Nov 2004

Highly frustrating for all, I see. No comment on Sven. Not in this form.

Re: "CoreTeam DON'T follow the current development process." Yes, I feel guilty. So, shall I quit my daytime job and work fulltime on TWiki? Who is going to pay my rent? I stated elsewhere that it is more work for me to work on two branches, and Crawford replied that there is no need to.

Not just CoreTeam, also developers do not follow the process. For example, we have a clear defined doc process (see DeveloperResponsibilities), and out of nowhere whithout reviewing/refining the process all docs end up in DEVELOP (from an incorrect base btw). So there are larger issues to resolve.

I understand why, but we made the mistake to start hammering on DEVELOP right away, now we have already a big gap. A more sensible approach would have been to initially make a few changes only, and merge them into MAIN, so that everyone has time to get adjusted to the new model.

-- PeterThoeny - 21 Nov 2004

In defence of Peter, I assured him that I would merge changes on MAIN across to DEVELOP, so he is only doing what we agreed.

In defence of developers, the documentation checkin was done by Sven (a core team member) taking the initiative. As I understood it he was trying to fulfil his role as develop branch facilitator, and facilitate the development of documentation on branches rather than being locked to a single version. IMHO this action was very sensible, to avoid polluting the documentation in TWiki with DEVELOP changes, but that's just my opinion.

Perhaps a less energetic approach to changes on DEVELOP would have been more successful. But I doubt it. The bottom line here is not the size of the changes in DEVELOP - which are mostly simple refactorings - but the nature of one specific change, namely the rendering order fix. We would be in this same situation now if I had drip-fed the changes into DEVELOP and waited patiently for merges - except, of course, that it would have been much more time-consuming for all to do that. I suspect it would have been mid-2005 before DEVELOP got to the point it is at today if we'd followed that process.

Enough bitching. I would rather focus on how we move ahead from here.

-- CrawfordCurrie - 21 Nov 2004

Let be continue the bitching a little more. I just can't refuse attacking a straw-man argument.

Peter, I understand that you have a daytime job. Guess what? So do I. Check RafaelAlvarezWouldLikeToCheckIn to know what I do for a living. Now, exactly what does it has to do with following or not following the development process?

If your reason is that you need those patches in MAIN to get your installations at WindRivers running, then SvenDowideitare is fully justified in his response. If that's not your reason, please state it here so we can understand each other, an vent away all the frustration so we can just move forward.

As for the documentation, why you think that the documentation should not be in the DEVELOP branch? The main argument for it is that it allows developers to change the existing documentation without screwing things or polluting the topic at twiki.org.

If you think that there are larger issues to resolve, please enumerate them in DeveloperResponsibilities or HowDoesTheDEVELOPBranchWork so we can work on then ASAP to the benefit of all.

Now, let's get pass the noise. Peter already said that he will check the changes in DEVELOP as soon as he can install it at work. Let's wait until he can describe whatever is broken so we can diagnose and fix it if the fix is in our hands.

-- RafaelAlvarez - 21 Nov 2004

Peter - do you want to go where we want to go? I am keen to read the PersonalRoadmapForPeterThoeny

-- MartinCleaver - 22 Nov 2004

Edit | Attach | Watch | Print version | History: r14 < r13 < r12 < r11 < r10 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r14 - 2004-11-22 - MartinCleaver
  • Learn about TWiki  
  • Download TWiki
This site is powered by the TWiki collaboration platform Powered by Perl Hosted by OICcam.com Ideas, requests, problems regarding TWiki? Send feedback. Ask community in the support forum.
Copyright © 1999-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.