Continuous Process Improvement

Summary of current proposals

Reflecting current state of agreement, as I read it (with Proposal #'s)

1. Add new state 'Confirmed' (web, done). Remap all currently 'Actioning' bugs to 'Confirmed' (script, ready). Proposal 1
2. Add new state "Being Worked On" (web, done) Proposal 1
3. Rename "Discarded" state to "No Action Required" (web, done) (script, ready)Proposal 1
4. Remove the "Requirement" priority (web, done). Map existing "Requirement" status to "Urgent". Proposal 2
5. Remove "Build" and "BugBase" domains (web, done). Remap bugs in these domains to "AppliesTo=Engine, Extension=Build"/BugBase as appropriate (script, ready). Proposal 3
6. Rename "MergeTo" to "TargetRelease" (web, done) (script, ready) Proposal 4
7. Move the Details area into the topic body and delete the CSS hiding the text body (web, done) (script, ready) Proposal 5
8. Add new field "Waiting For", which accepts a name, and is used to point to a reporter (if "Waiting for Feedback") or a developer (if "Being Worked On") (web, done) Proposal 6
9. Add field "Publicly Released In" (web, done) Proposal 7
10. Rename "Extension" field to "Component" (web, done) (script, ready) Proposal 8

-- CrawfordCurrie, ThomasWeigert - 19 Jan 2007

OK. Let's do it... we need somebody who has access to the bugs web...

-- ThomasWeigert - 19 Jan 2007

The script needs finishing (it doesn't do everything) and testing. I won't be able to get to that until Monday at the earliest, but that shouldn't stop anyone else....

-- CrawfordCurrie - 20 Jan 2007

All done

-- CrawfordCurrie - 23 Jan 2007

Proposal 1: Add new state

Background

There is some confusion over the meaning of some of the states in the bug DB. It would be good to get these cleaned up after 4.1.

> Steffen Poulsen wrote:
>>Thomas Weigert wrote:
>>>To me "Actioning" means somebody has grabbed this item and will solve it.
>>>"New" means that there are no pending questions but nobody has decided to
>>>work on it.
>>>
>>>The bugs state machine seems to be popping between "new" and "waiting for
>>>answer" for a while, then go into "actioning", then go into "waiting for
>>>release"....
>>
>>I second this view - to me "actioning" currently means "taken", "new"
>>means "available".
>>
>>Of course we can redefine this if best be.

Here are the state descriptions we have been working with for some time now (taken from the Bugs edit page):

• New the issue has just been found, and hasn't been analysed yet, or there is new data about an existing issue.
• Actioning the issue has been analysed, and either someone is working on a fix, or it is waiting for someone to fix it .
• Waiting for feedback the report was insufficient to analyse the issue, and more feedback is needed from the reporter. Once the reporter has added feedback they should flip the state back to "New". If feedback is not received within 30 days, the issue will be discarded.
• Waiting for release the issue reported has been fixed and is queued for release. See the MergeTo field to see whether it is queued for the next patch, minor, or major release. If the reporter isn't satisfied with the fix, they should flip the state back to "New" and explain why.
• Closed the issue has been fixed to the satisfaction of the fixer, and has been through the "Waiting for release" state, and is now released to end users. If the reporter isn't satisfied with the fix, they should flip the state back to "New" and explain why. Note that bugs reported against new code (code that has not been released yet) or Extensions (Plugins etc) may jump straight to "Closed" without going through "Waiting for Release" first.
• Discarded the issue was already fixed, or it's a duplicate, or a RTFM, or could not be reproduced and feedback was never received.

So as far as I can see, no redefinition is required. Originally I intended "Actioning" to be as Thomas describes, but found that once a bug flipped to "Actioning" people would ignore it, as they assumed it had been "assigned" to someone. "Actioning" is a poor choice of a name for the state, but at this point I really don't want to change it. Perhaps after 4.1 we should bite the bullet and rename it everywhere to "Waiting for a fix".

At the same time we should rename "Discarded" to "No Action Required", removing the negative connotations. No-one likes to feel they have been "Discarded".

Something else I'd like to address is some way to determine the releases that a bug is fixed in. At the moment it's not obvious after a bug is closed what release numbers actually have that fix. With a bit of arithmetic, the SVN number tells you which minor/major release, but not which patch(es). I'd like to have that information on a bug report page (though it could be determined by a cron job in the background if necessary). Any ideas/solutions?

-- CrawfordCurrie - 20 Dec 2006

Proposal detail

Status	Meaning
New	The issue hasn't been analysed yet, or there is new data about an existing issue.
Waiting for Feedback	the report was insufficient to analyse the issue, and more feedback is needed from the reporter. Once the reporter has added feedback they should flip the state back to "New". If feedback is not received within 30 days, the issue will be closed as "No Action Required".
Confirmed	The issue has been analysed, and the priority has been confirmed (or changed) by a developer. The item can henceforth be classified regarding release blocking status etc. Someone ought to take care for the item and fix it.
Actioning	Someone (should be clarified in the details) is currently working on the item. Fellow developers should either be patient, or contact the person who's actioning, or be prepared to do duplicate work. Items should be moved automatically back to either New or Confirmed if they have been in Actioning for 30 days, or shorter if we are approaching a release date, up to the release manager's decision. So if you want to keep the claim on the Item, make sure to take a compass with you before you enter the rain forest.
Waiting for Release	The item is fixed and has to be mentioned in the release notes.
Closed	The item is fixed but needs not to be mentioned in the release notes.
No Action Required	The item doesn't need to be fixed.

-- HaraldJoerg - 21 Dec 2006

Discussion

I think it is a bad idea to have the same state name for

• I commit to fix this bug, and
• I hope somebody else will fix this bug

Please, please introduce a state for each, albeit I do not understand the difference between "new" and "I hope somebody will fix this".

-- ThomasWeigert - 20 Dec 2006

I second Thomas and Steffen to re-define the meaning of the "Actioning" state after 4.1 since this has been confusing me a lot. IIRC it is sourceforge which has a status of "Assigned", which means assigned to someone who commits to be working on the bug. I wouldn't even go as far as claim "fixing", because sometimes one might need to hand over fixing to someone else. We need a state which saves us from duplicating work if two or more of us are working on a bug which is in "Actioning" state, and we need a state to make sure that a bug stays "Actioning" forever (which has happened recently).

I haven't seen any advantage of the "Actioning" status in its current meaning recently. If I have time to spend on a bug, I don't care whether it has been analysed or not. Many times, as soon as I had analysed it, the fix was rather simple anyway. So I'd opt for abandoning "Actioning" in its current meaning, but to avoid misunderstandings we could call the new state "Assigned".

Renaming "Discarded" to "No Action Required" is fine for me.

Finally I agree that determining the release would be helpful. For minor/major I guess that a post-commit script could do that automagically, but for hotfixes it would be a manual job for whoever creates the hotfix, right?

-- HaraldJoerg - 20 Dec 2006

Aw heck, it's clear that Kenneth at least misunderstood the language I used in the description of the Actioning state.

• Actioning the issue has been analysed, and either someone is working on a fix, or it is waiting for someone to fix it .

This means "the analysis is clear, and this is just waiting for someone (anyone) to feel sorry for it and fix it", and not "the issue is assigned to someone specific".

We have learnt through painful experience that assigning issues in the TWiki project does not work. Too many times we have been left in limbo, with an issue that everyone thinks X is working on, but X has actually lost interest in TWiki and wandered off, or is just sitting on the issue, or is lost in the Borneo rainforests. If you want an example of this, look at the "Assigned Questions" in the Support web. I repeat. We tried it and assignment does not work. Let's not go there again, please!

The value of distinguishing between "New" and "Actioning/Analysed/Waiting for a fix" is that when someone visits the Bugs web they can see at a glance what issues have not yet been looked at by a developer. Until an issue is analysed, there is no way to know whether it should block a release or not. When it has been analysed, the steps required to fix it are clear, and anyone can pick it up and fix it. This distinction is crucial to the Release Manager in deciding whether it is safe to release or not. You should never release with un-analysed Urgent or Requirement issues.

Given the ratio of bugs to active developers, I don't see duplication as a big issue, but if you want to avoid it then add text to that effect that you are working on it in the Details section. But I advise against it unless you really are working on it. There are several reports in the DB that have "I will do this" notes in them, that I am concious of avoiding fixing, because I think someone else has claimed them.

-- CrawfordCurrie - 21 Dec 2006

Crawford, your point is a good one, but we need to have another state to express it. Everybody interprets "Actioning" as "I am signing up for fixing this, you don't need to worry about this any longer".

If you want a state different from "New" which means "we checked this out and did all the ground work, but now, please somebody have pity and start working on it", then let's do that.

Having "Actioning" do double duty is not a good idea...

-- ThomasWeigert - 21 Dec 2006

... either someone is working on a fix, or it is waiting for someone to fix it ...

It is exactly this or which we should get rid of. I didn't know that you've tried assignment and it failed, sorry for that. Nevertheless I think that just throwing it all into one bucket is even worse.

So let me try a new proposal:

Status	Meaning
New	The issue hasn't been analysed yet, or there is new data about an existing issue.
Waiting for Feedback	the report was insufficient to analyse the issue, and more feedback is needed from the reporter. Once the reporter has added feedback they should flip the state back to "New". If feedback is not received within 30 days, the issue will be closed as "No Action Required".
Confirmed	The issue has been analysed, and the priority has been confirmed (or changed) by a developer. The item can henceforth be classified regarding release blocking status etc. Someone ought to take care for the item and fix it.
Actioning	Someone (should be clarified in the details) is currently working on the item. Fellow developers should either be patient, or contact the person who's actioning, or be prepared to do duplicate work. Items should be moved automatically back to either New or Confirmed if they have been in Actioning for 30 days, or shorter if we are approaching a release date, up to the release manager's decision. So if you want to keep the claim on the Item, make sure to take a compass with you before you enter the rain forest.
Waiting for Release	The item is fixed and has to be mentioned in the release notes.
Closed	The item is fixed but needs not to be mentioned in the release notes.
No Action Required	The item doesn't need to be fixed.

-- HaraldJoerg - 21 Dec 2006

OK, that's a good proposal. As reluctant as I am to add new states to the lifecycle, the weight of opinion is against me.

Issues currently in "Actioning" state need to be moved to "Confirmed" state. They can subsequently be moved to "Actioning" as people commit to tending them. The searches and the documentation in the Bugs web need to be updated.

I won't be online again until after Chrsitmas, so if Steffen/Thomas/Kenneth agree (or fail to disagree) could someone please add a Bug against the BugBase to that effect? Marked "Confirmed", of course

-- CrawfordCurrie - 22 Dec 2006

I think it was so obvious that Actioning meant taken that noone read the text in the bugsweb. I wonder who wrote that crazy meaning of Actioning in the first place. (That was CC, see above.)

I have been trying to work as a release manager for 4.1 and have done a lot of walkthroughs and following up on even very old bug items.

Now I have completely lost that overview after Crawford decided to change state of so many bug items. I think the major part of developers now have no clue if an actioning bug item should be touched or not.

I can support Harald's proposal even though I do see the added value in the Confirmed state. I think what we have today is fine as long as Actioning means "Actioning". But if people want the Confirmed state then fine. I just think it will not be used much in practical. 99% of bug items are opened by developers and can be put in Confirmed right away.

I also agree that in an open source community bugs gets TAKEN not ASSIGNED. So the process that a developer TAKES ownership by setting a bug item in Actioning is the right approach. And the purpose of this state should be the signal that other developers need not worry about it.

I do not know what to do with the bug reports now. We had like 10-12 actioning bug items and now there are 21. Are people working on them? Should we flip them all back to new?

We cannot wait till post 4.1 release to settle this. I need a tool I can work with NOW to get 4.1 released. And I did not need this extra noise right now.

-- KennethLavrsen - 23 Dec 2006

I observed another unpleaseant BugsPingPong in Bugs:Item2650: A developer closed a bug as the reporter had not provided a test case to verify the bug and the developer could not reproduce the bug. Upon that the reporter was upset, reopened the bug, and the ping pong ensued.

Both sides had a point:

• The developer did not want bugs around that he knew nobody would fix until somebody could come up with a test case.
• The reporter wanted it recorded that he had observed a bug and did not want that information to be lost.

To avoid this, we might add yet another state: Limbo with the meaning "I have examined the bug but could not reproduce it and there was no test demonstrating the presence of the bug". Such state would avoid having to close the bug but would not clutter up the bug pipe line with irreproducible reports. A bug can come out of Limbo if it is reproducible.

-- ThomasWeigert - 03 Jan 2007

That would be the first bug tracker I have seen with a Limbo state. We have the "waiting for feedback" which is exactly meant for this sort of situation.

-- KennethLavrsen - 03 Jan 2007

There is a subtle difference... waiting for feedback is to ask questions during the work on the bug or trying to identify the bug... limbo is for what the developer wanted to do in the case quoted above: get the bug out of the pipeline until it is recognized as a bug. No big deal, but I think whatever we can do to avoid conflicts between reporters and developers, the better...

-- ThomasWeigert - 03 Jan 2007

Pointless ping pongs are very frustrating, so anything that helps solve this issue is good. This can be done simply by policy ("developers should not discard an unreproducable bug if there is reasonable believe that it is a bug"), or with a new state as Thomas suggest. I kind of like a new state, although we might end up with a big dev/null pile.

-- PeterThoeny - 04 Jan 2007

As the International Theological Commission is now expected to recommend to deamphasize the concept of Limbo (at least as the explanation of the eternal fate of unbaptized babies), it is time to put the concept to new use... but you are right, this is probably a euphemism also for /dev/null, just a more polite one...

-- ThomasWeigert - 04 Jan 2007

I don't think the Limbo state is useful (though I like the name). If we can't explain a 6 state lifecycle to a newbie, we have even less chance with a 7 state one with even more subtle distinctions.

The conflict over the Actioning state arose after two different audiences applied their own interpretation of the same state description.

Historical aside: The state name "Actioning" originally meant "someone is fixing it". We changed the description after too many issues fell into the black hole of being assigned and ignored. I changed the description but at the time I didn't have login access to d.t.o, and was unable to find anyone who did who was prepared to actually rename the state.

The first audience, developers used to working with bug DBs, are used to the process of analysis, whereby a reported bug is nailed down to the point where a fix is possible. That means reproducing the problem, researching testcases, etc. Reasonable belief is not a valid metric; just ask any two competing religions what a "reasonable belief" is.

The analysis process inevitably involves some ping-pong with the reporter, which is why we introduced the Waiting for Feedback state. The policy of not discarding any issue that the reporter has been given a fair chance to clarify (1 month) is IMHO a reasonable one, and addresses what happened with Bugs:Item2650.

The conclusion of the analysis process is an analysed state (the current meaning of Actioning). This state is essential for anyone who actually fixes problems in the bug DB.

The second audience is managers, such as the Release Manager. A manager wants to know that something is being done about a bug, so interprets "Actioning" as meaning that someone has committed to doing something about it. Because of the historical almost total lack of commitment from the TWiki community, we deliberately never enshrined this state in the bug DB, though Harald is now proposing adding it (see the discussion above).

I have an ongoing concern about adding a committed state in the TWiki project. As soon as someone commits to fixing a bug, everyone else will go "hands off". Too many people have in the past committed to fixing something, only to have it effectively shelved. In that sense I suspect the term Limbo will prove to be better applied to this state. Even so, I am prepared to add the state in a spirit of "let's see what happens".

-- CrawfordCurrie - 04 Jan 2007

I do not recognize the situation you describe.

Both developers and the release manager need to know if a bug item is being worked on.

The developers to know if a peer is already working on it to avoid double work.

The manager to know what to push for. And actually I see the developer need as the important one.

I do not believe I remember any developer that said anything else than what I have said all along. All that have expressed oppinions have said that they saw "actioning" as "taken".

I think we have the exact right number of states that we need. I do not see the a great need for an analysed state. Quite many bugs are raised by the same person who intend to fix it. All you need then is to open it as actioning and take it straight to closed or waiting for release.

I agree to rename Discard to something nice. "No Action Required" is fine.

I would however like to see a field with the developer name so we can see who is actioning. This would be a "I have taken this and my name is" field. In an open source project I 100% agree that you get nowhere with assigning bugs to anyone else than yourself which is why assigned is a wrong name for this state.

-- KennethLavrsen - 04 Jan 2007

Probably because you don't analyse that often. Analysis is my major activity in the bugs DB, and I have to know when something has not been analysed yet. I don't care if someone claims to be working on it. As I said, I don't believe them anyway. However I don't have a problem with a WorkingOnIt field that accepts a list of developer names, as long as the existance of that field doesn't block me from checking in even though my name isn't on the list.

-- CrawfordCurrie - 05 Jan 2007

Proposal 2: Drop the "Requirement" priority

There is no clear distinction between "Requirement" and "Urgent".

Proposal detail

Drop the "Requirement" priority.

-- CrawfordCurrie - 04 Jan 2007

Discussion

Agree

-- KennethLavrsen - 04 Jan 2007

Agree 2

-- PeterThoeny - 05 Jan 2007

Proposal 3: Drop unneccessary domains

There are two excess problem domains, Build and BugBase.

Proposal Detail

Drop Build and BugBase and use the Extension field instead.

-- CrawfordCurrie - 04 Jan 2007

Discussion

I would like to keep a problem domain for fixes / enhancements in the test environment (unit tests etc.). Unlike Extensions, changes in test cases have no impact for the end user, and as soon as an Item has been confirmed to belong to a test case (and not to the code which it tests) it should not block a release. Recently I have been (ab)using the "Build" domain for that purpose since I think that developing and running tests are part of the Build process, in a general sense.

-- HaraldJoerg - 04 Jan 2007

I don't think you have been abusing it because that is what Build was intended for. Tests are part of the build system.

I didn't make myself quite clear above. The reason I want to drop Build is that it covers a lot that is specific to the Engine, and AppliesTo=Engine and Extension=TestCases / Extensions=BuildScripts is a better way to express that. It makes writing searches in the DB a lot easier if we reduce the number of top level domains, at the cost of some loss of control over the domain names. I don't believe you lose anything this way, in fact I think you gain all round.

Perhaps 'Extension' is the wrong name for that field, but I can't think of anything much better.

-- CrawfordCurrie - 05 Jan 2007

Proposal 4: Rename, and change the usage of, the MergeTo state

The MergeTo state is used to collate the contents of the next release; if a report is "Waiting for Release" then this field relfects the targeted release. The field is currently underused, is misnamed, and is sometimes confusingly set by reporters and even developers who don't read the descriptions.

Proposal Detail

The MergeTo field should be renamed "TargetRelease". The meaning should be as follows:

Used only by developers and the Release Manager for planning and recording the contents of releases. This field has different meanings depending on the State:

• If the State is New or Waiting for Feedback then the value of this field is irrelevant.
• If the State is Waiting for Release then this report describes a fix *planned to be incorporated in the next release of the module identified by AppliesTo.
• If the state is anything else then the Release Manager wants this report to be fixed for the next release.
  • n/a means not confirmed, or refers to an extension that is not part of a standard TWiki release, or it hasn't been decided which release to target yet, or the fix was required because something broke in a prior bugfix.
  • patch refers to the next bugfix release, and should be used for urgent bugfixes and minor corrections only.
  • minor refers to the next maintenance release, and may be used for minor functionality improvements and refactorings.
  • major refers to the next full release, and should be used for all functionality changes, and any deprecations.

It would be neat if we could restrict the use of this field to the release manager.

-- CrawfordCurrie - 04 Jan 2007

Discussion

The reality of current developement and what I think is the right way to continue is.

There are TWO branches alive on SVN.

• MAIN which targets the next minor OR major release per plan. We will never have resources to both work on a minor and major in parallel.
• Current release. For patch fixes.

The idea is that bugs are always fixed in MAIN. Enhancements are ONLY made in MAIN. When the developers OR the release manager finds it applicable the same bug fix is applied to the patch release branch.

So what we really need

• Before the fix is done the developer AND release manager can indicate if the bug should be in next patch release.
• After the fix is done the SVN committer adjusts to indicate which of the two the fix is committed to.

This way we can easily create the release notes for both patch releases and minor/major.

-- KennethLavrsen - 04 Jan 2007

OK, I changed the text to reflect your points. Note however I will never merge to a patch release branch, unless I am myself the release manager. I think it's a really bad idea (there is far too much risk that a developer will decide that their pet hack just has to be in the next release, and they would be hard to stop)

-- CrawfordCurrie - 05 Jan 2007

Proposal 5: Move the Details area from the form to the topics content area

Did never understand why the explaining details had to be within the form, where it would be much more TWiki-like to have them in the topic body. This also would have the advantage that the standard diff-view could provide more useful history insight.

Proposal Detail

Any further arguments necessary?

-- FranzJosefSilli - 04 Jan 2007

Discussion

I think this will enhance the usability of the bugs web substantially. We can use the full screen, use comment boxes, write test cases in the topic, etc. Please add an included header on top that summarises the bug item (heading with summary, followed by priority and state info)

-- PeterThoeny - 05 Jan 2007

Agreed, if it can be done. I think it's very difficult to do that, though, from the current setup, which is the main reason I never tried myself. It would require decoding the form field value, and embedding it in the topic body, quite aside from the required changes to searches. I assume a form field is used to simplify passing parameters to the various queries. I'm not really sure.

-- CrawfordCurrie - 05 Jan 2007

I don't think there is any query involving the description field. It probably would be fine to just start using the textarea for the description (and, of course, to remove the silly CSS that hides the text area in view mode).

-- ThomasWeigert - 05 Jan 2007

Thanks for Crawfords script this is realized now, although there additionally should be added an item summary at the top as proposed by Peter. An INCLUDE at the top should be enough. -- Great work Crawford! Will have a deep look at your script some day to see how you did it.

-- FranzJosefSilli - 23 Jan 2007

Proposal 6: Add "who from" field to "Waiting for Feedback" state

Often, waiting for feedback is not addressing the community at large, but a particular developer or reviewer. There is no way of indicating that in the bug. I propose to add a field where one can indicate who the question is addressed to.

Proposal Detail

Example: Say I have a question to CC. By indicating that in the "who from" field, if he has a search that looks for that field, he could be alerted as to this question. Currently I have to email him to get his attention.

-- ThomasWeigert - 06 Jan 2007

Discussion

Good idea. If we use "Waiting For" it can apply to the "Waiting for a fix" state as well.

-- CrawfordCurrie - 07 Jan 2007

This is really important. I just noticed that Michael took out a patch to Bugs:Item2650 because it had a small, but unwanted side effect. As I had not filed the original bug report, the state change did not show up in my searches (I have a cockpit where I look for the defects that I report). So it was just sitting there in "Urgent"; I could have long addressed it had I been aware of it.

We need a field that one can search for that brings a bug to one's attention.

-- ThomasWeigert - 14 Jan 2007

Proposal 7: Add field "Publicly Released In"

So we know what release a fix was in. Take a release name, same set as in "Seen in release"

-- CrawfordCurrie - 08 Jan 2007

Discussion

Proposal 8: Rename "Extension" to "Component"

So it can be used for components of the Engine, such as BugBase, BuildSystem and UnitTests

-- CrawfordCurrie - 08 Jan 2007

Discussion