Refactored from PleaseCreateNewWeb after a conversation I had with PeterThoeny in which he said he'd favour new categories over a new web.

-- MartinCleaver - 6 Jul 2003

How about a single extra category

(...) We could certainly add a new category called something like DesignBrainstorming or even NextGeneration

-- JohnTalintyre - 19 Jun 2003 (... I've split John's comment but hopefully not changed his meaning)

Martin's Initial Suggestions For Overhaul

How about:

• add flag: AlertCoreTeam
  • Most of the time, conversations on TWiki.org can proceed without the involvement of the CoreTeam. Occasionally though, (for instance when a request is made for changes to be made to WebForms on the site), the core team have to get involved. Having a flag that others could tick would mean that the core team could instantly get a list of things that need their attention. The core team would simply untick the request once they had dealt with it.
• add TopicClassifications: CodingPrinciple, ArchitecturalPrinciple, UsabilityPrinciple
  • CodingPrinciple. E.g. items discussing the virtues of rules such as 'no subs longer than 100 lines'; always 'use strict'; always 'use my'.
  • ArchitecturalPrinciple. E.g. items discussing the virtues of rules such as 'someone should be reading all code contributions and spotting and refactoring duplications',
• add TopicClassifications: FeaturePatchAvailable, FeaturePatchApplied, FeaturePatchRejected
• split FeatureToDo into FeatureToDoUnassigned, FeatureToDoAssigned
  • This will allow coders who have some spare time to pick an item, claim ownership. Once it is ready it them becomes FeaturePatchAvailable
• remove ProjectGroup
  • It is seldom used consistently and therefore of little practical use.
• and either
  • replace category ImplementationDate with categories: ImplementedInAthens, ImplementedInBeijing, ScheduledForCairo, ScheduledForDakar,
    • have a rule that on a new release:
      • Old issues are moved back to ImplementedInXXX
• or (as in PleaseCreateNewWeb)
  • rename web Codev to 'Current', and create new webs CodevArchive, ScheduledForCairo, and ScheduledForDakar
    • have a rule that on a new release:
      • Old issues are moved back to web CodevArchive
• add field: PageOwner, defaulting to page creator - person responsible for summarising and concluding or for passing that responsibility to another.
• add field: PageCreator, the name of the person who created the topic.

-- MartinCleaver - 20 Jun 2003

Categorisation System Requires "mothering"

The underlying issue is not that we need more categories or a clearer defintion of the categories but that many people don't use the mechanism and use it consistently.

The 'banner' on the Plugins web has a means of searching catgories - see WebPreferences .... Set WEBTOPICLIST. But the topics still have to be categorized. If the topic creators don't do it there needs to be a "Web Mother" - human or automated - that does it. (Automated - puts the topic in the same category as its parent, perhaps?)

Maybe, just maybe, if the 'banner' has those extra items it will encourage people to be more aware of the categories.

Slightly off-topic: I'm curious how other wikis implement categorization. PPR seems to allow a topic to be in multiple categories, bioth technically and as the human-interface process. Using webforms for categories precludes using them for other application-specific things.

-- AntonAylward - 21 Jun 2003

Do the categories match the core team's expectations ?

As for the Web Mother thing - again this is a CoreTeamNominationDiscussion process that the core team are still thinking about. It's a role that Mike Mannix used to do and that Peter T has not reassigned since Mike's mysterious disappearance.

CoreTeam - how about the categories I have listed above?

-- MartinCleaver - 30 Jun 2003

Change Banner to match Categories/How they're used?

Re: Anton's

• The underlying issue is not that we need more categories or a clearer defintion of the categories but that many people don't use the mechanism and use it consistently.

I agree people don't use it and don't use it consistently. I think this is partly due to the existing system being inadequate. That's why ProjectGroup should go and why I've split up some of the existing categories so they become unambiguous.

I also agree that the top-level banner would need changing to reflect the catagories.

-- MartinCleaver - 06 Jul 2003

Sample Implementation of Martin's Schema - Messy?

Based on your description I've knocked this up, and IMO it's messy. I've deliberately split the form into separate lines to show how it might look:

Name	Type	Size	Values	Tooltip message
AlertCoreTeam	select	1	No, Yes	Does this involve a change to TWiki's core distribution?
TopicClassifications	select	1	Select one..., TWikiDeployment, TWikiVsOtherProducts, TWikiAddOnProduct, TWikiDevQuestion, FeatureBrainstorming, FeatureEnhancementRequest, FeatureHack, FeatureUnderConstruction, FeatureDone, FeatureNotSuitable, BugReport, BugAssigned, BugResolved, BugDuplicate, BugRejected, DocRequest, DocsToDo, FeatureDocumented, DefineTerm, AdminTopic, GatewayTopic, CodingPrinciple, ArchitecturalPrinciple, UsabilityPrinciple, FeaturePatchAvailable, FeaturePatchApplied, FeaturePatchRejected, FeatureToDoUnassigned, FeatureToDoAssigned FeaturePatchAvailable
Implemented...	select	1	N/A, TWikiAlphaRelease, TWikiBetaRelease, TWikiProductionRelease, TWikiRelease01Feb2003, TWikiRelease01Dec2001, TWikiRelease01Sep2001, TWikiRelease01Dec2000, ImplementedInAthens, ImplementedInBeijing, ScheduledForCairo, ScheduledForDakar, ImplementedInXXX
PageOwner	text	40	%WIKIUSERNAME%
PageCreator	text	40	%WIKIUSERNAME%

Slightly Simpler Version of Martin's Schema - Still Pretty Nasty?

I've tried something slightly better as well IMO, but it's still pretty horrendous to be faced with:

Name	Type	Size	Values	Tooltip message
AlertCoreTeam	select	1	No, Yes	Does this involve a change to TWiki's core distribution?
TopicClassifications	select	1	Select one..., TWikiDeployment, TWikiVsOtherProducts, TWikiAddOnProduct, TWikiDocumentationRelated FeatureDevelopment BugReport TWikiDevQuestion
FeatureStage	select	1	Select one..., FeatureBrainstorming, FeatureEnhancementRequest, FeatureToDoUnassigned, FeatureToDoAssigned FeatureDone, FeatureNotSuitable
BugReportStage	select	1	Select one..., BugReport, BugAssigned, BugResolved, BugDuplicate, BugRejected
ImplementationStatus	select	1	Select one..., UnderConstruction, PatchAvailable, HackAvailable, PatchApplied, PatchRejected
ImplementationComment	text	40	..	Reason patch reject, or link to discussion page
DocumentationType	select	1	Select one..., DocRequest, DocsToDo, FeatureDocumented, DefineTerm, AdminTopic, GatewayTopic, CodingPrinciple, ArchitecturalPrinciple, UsabilityPrinciple,
ImplementedIn	select	1	N/A, TWikiAlphaRelease, TWikiBetaRelease, TWikiProductionRelease, TWikiRelease01Feb2003, TWikiRelease01Dec2001, TWikiRelease01Sep2001, TWikiRelease01Dec2000, ImplementedInAthens, ImplementedInBeijing, ScheduledForCairo, ScheduledForDakar, ImplementedInXXX
InterestedParties	text	40	%WIKIUSERNAME%	People who have an interest in this page - page creator, "owner", feature implementor, feature user, etc

What this is symptomatic of is an inability to have a random selection of fields. For example a BugReportForm, a FeatureStatusForm, and so on. You can have a selection of forms available (as is at the moment), but you can only add one form. For example it makes sense for Features & Bugs to share implementation issues, but not on the same page.

All this actually boils down to a normalisation issues, which are pretty well understood - TWiki forms are 1st normal form, with the key being the topic name. (Essentially a TWiki page with a collection of forms actually forms a view, and the UI is a form of view as well)

As a result of this however, I think unless the Forms are made richer this will remain a problem no matter what classifications we choose. As a result I'd be more in favour of a simplification of the current categories rather than an explosion/expansion.

A Further Significantly Simpler Version of Martin's Schema - Still complex/High bar for user?

A possible simplification of the above might be:

Name	Type	Size	Values	Tooltip message
TopicClassifications	select	1	Select one..., TWikiDeployment, TWikiVsOtherProducts, TWikiAddOnProduct, TWikiDocumentation, FeatureDevelopment, BugReport, TWikiDevQuestion	Please select a primary classification
DocumentationType	select	1	Select one..., NotApplicable, DocRequest, DocsToDo, FeatureDocumented, DefineTerm, AdminTopic, GatewayTopic, CodingPrinciple, ArchitecturalPrinciple, UsabilityPrinciple	If this is not feature development please fill this in
DevelopmentStatus	select	1	Select one..., NotApplicable, ---FeatureStatus, FeatureBrainstorming, FeatureEnhancementRequest, FeatureToDo, FeatureAssigned, FeatureDone, ---BugStatus, BugReport, BugAssigned, BugResolved, ---CodeActivity, UnderConstruction, PatchAvailable, HackAvailable, PatchApplied, ---Rejected, FeatureNotSuitable, BugDuplicate, BugRejected, PatchRejected	Status of development regarding feature or bug from conception to application/rejection
ImplementationComment	text	40	..	Reason patch reject, or link to discussion page
ImplementedIn	select	1	NotApplicable, TWikiAlphaRelease, TWikiBetaRelease, TWikiProductionRelease, TWikiRelease01Feb2003, TWikiRelease01Dec2001, TWikiRelease01Sep2001, TWikiRelease01Dec2000, ImplementedInAthens, ImplementedInBeijing, ScheduledForCairo, ScheduledForDakar, ImplementedInXXX	Version this includes this feature/bug fix, (or is scheduled for this feature - where appropriate)
InterestedParties	text	40	..	People who have an interest in this page - page creator, 'owner', feature implementor, feature user, etc

What this looks like.

In practical terms that looks like this:

DemoForm
TopicClassifications	Select one... TWikiDeployment TWikiVsOtherProducts TWikiAddOnProduct TWikiDocumentation FeatureDevelopment BugReport TWikiDevQuestion
DocumentationType	Select one... NotApplicable DocRequest DocsToDo FeatureDocumented DefineTerm AdminTopic GatewayTopic CodingPrinciple ArchitecturalPrinciple UsabilityPrinciple
DevelopmentStatus	Select one... NotApplicable ---FeatureStatus FeatureBrainstorming FeatureEnhancementRequest FeatureToDo FeatureAssigned FeatureDone ---BugStatus BugReport BugAssigned BugResolved ---CodeActivity UnderConstruction PatchAvailable HackAvailable PatchApplied ---Rejected FeatureNotSuitable BugDuplicate BugRejected PatchRejected
ImplementationComment
ImplementedIn	NotApplicable TWikiAlphaRelease TWikiBetaRelease TWikiProductionRelease TWikiRelease01Feb2003 TWikiRelease01Dec2001 TWikiRelease01Sep2001 TWikiRelease01Dec2000 ImplementedInAthens ImplementedInBeijing ScheduledForCairo ScheduledForDakar ImplementedInXXX
InterestedParties

Whether this is actually better I'm not convinced. One thing I am sure of - unless people are forced to fill in the form and put sensible values, then the more complex the form is the more useless it is. (One way may be to clone the form infomation from one topic to another at topic creation time? (ie copy the parent's classifications in, or a templatetopic's classifications.)

What a further subset/simplification would be I'm not sure - I do know that the webforms the above tables generate are FAR from friendly. I suspect something "clever" could be done with respect to implementation. (Code Base vs release might be the way forward and defined using TWikiVariables perhaps? )

-- MichaelSparks - 10 Jul 2003

Need to live with complexity in order to get things done ?

Thanks for working on this Michael, I do appreciate it. I think there maybe two things to consider here:

1. TWiki's needs in the short-term
2. What we can accomplish in the long term and for what cost.

It boils down to your comment:

• ...I think unless the Forms are made _richer this will remain a problem ..._

I think ultimately needing AllowMultipleForms is a requriement, I've wanted this for other purposes before anyway. It will be required, for example, if the CairoRelease page is to be constructed from a search where the each feature comes from its own topic and each topic has of one of the above complex types.

In the short term I think we live with the complexity; I suspect it is better to allow people to leave blank fields that are inappropriate than not capture the detail we need to be able to better manage the project.

-- MartinCleaver - 10 Jul 2003

Trying to use WebForms to Implement Workflow - Not designed for that?

I think the problem with forms on Codev is that what we're realy trying to use them for is to implement a workflow when what they currently do best is topic catagorisation that stays fairly static.

What we realy need is a workflow system, perhaps a way of mapping changes beteen states/forms, that has it's interface when viewing topics (although it would probably still be available while editing) and that reloads the page with updated forms.

-- SamHasler - 11 Jul 2003

And in turn this is related to something that Anton said earlier: that #FormsShouldBeSeparateToCategories .

Given that they are not, that we've established more requirements, how do we use what we have today to accomplish our end goal of improving our development process? Can we use Michael's scheme?

-- MartinCleaver - 12 Jul 2003

---

Like so many things, its probably going to get worse before it gets better.

Michael pointed out, amplifying my earlier comment,

• unless people are forced to fill in the form and put sensible values, then the more complex the form is the more useless it is.

and as Sam point out, there is a lot of "workflow" in the outlines above.

But the "root cause" runs deeper. The Codev web is a grab-bag of many things. Yes, they need to be categorized, but they probably need to be categorized in a heirarchical fashion. As the tables above show, there are some very distinct and orthogonal categories.

One (the many) way of looking at things is Past Present Future
I'm presenting this not as a "make it so" proposal but to illustrate another orthogonality, another way of paritioning topics.

Past

• How things were
• What design decisions were made and the discussion surrounding them
• Stuff that is obsolete such as bugs and fixes in old baseline releases

Present

• The current design
• The current documentation
• Discssion relevant to the current design.
• Bugs and fixes since the last baseline release

Future

• Proposed design for the next - and only the next - release
• Design decisions relating to this

This partition is "one of many", remember. In a resource rich environment the demarkations could be strong enough to justify seperate webs - certainly so if we have heirarchical webs.

It also highlights that there are things in the current Codev grab-bag that don't fit this model. The point here is not that this model is inadequate but that the present Codev has turned into a grab-bag. The same lack of "enforcement" of categorization we are discussion applies.

Even this topic is an example - its really a 'meta-discussion' not a Codev discussion.

I "grew up" in a resource rich UNIX enviroment. If I ever needed to reorganise space - project or personal - I could create create directories, move files, set up links. I'm sure just about everyone who has dealt with non-trivial projects has done something similar. I'm also sure those of us who understand hierarchal filing laugh at the non-technical people who keep all their documents and spread-sheets in one humoungous "My Documents" folder; no partitioning by client, project or anything else. We recognise that they are failing to use the capabilities of the system.

The capabilities we don't have here determine how we can organise and partition the topics:

• Ability to create webs is restricted
• Only one "category" per topic (I've discussed this elsewhere)
• Only one form per topic
• Category is not an orthogonal metadata attribute of the topic
• No workflow process for the activities that occur in Codev

The first and second are not technical impediments, they are "decisions on the part of management". In all honesty, considering the volume and diversity of Codev and how its use has "evolved" away from simply "code development", and considering threads such as this, I can't understand the reluctance to create a new web.

In one sense a web IS a categorization. Unlike forms-based categorization it does force the user to make use of the category. At one level it also addresses Michael's very valid observation about complexity. More forms and richer forms means more decision points, as as Martin points out, they are likely to be left blank. If one of the uses of categorization is searching, then this is unhelpful.

Tell me again why there is such reluctance to create new webs, there's something about that attitude I don't understand. Even without wearing my "Spock-ears" the adamant reluctance seems illogical. If there is a case for more categories there is a case for more webs.

• One very good reason for when a separate web is preferable to a separate category is when you have the same topic existing in both contexts but where the content of that topic should depend on the context (WhatDifferenceBetweenATopicAndAWeb, ImportingContexts); this is a clear candidate for a copy rather than a conditional in the subclass (OO advocates know that branching on type is symptomatic of a poorly designed Object orientation). -- MartinCleaver - 12 Jul 2003
  • By way of a (bad) example, if you look at WindowsInstallCookbook (search 'minor changes'), you'll see there is a reference saying you need to do something specific to for pre-Cairo. -- MartinCleaver - 12 Jul 2003
  • Imagine a set of InstalllationInstructions that vary tremendously depending on whether we are talking about Beijing or Cairo, we'd want Beijing.InstallationInstructions and Cairo.InstallationInstructions not Codev.BeijingInstallationInstructions & Codev.CairoInstallationInstructions. -- MartinCleaver - 12 Jul 2003

-- AntonAylward - 12 Jul 2003

A Working Implementation of the Suggested "Simpler" New Categories - As a TWiki Application

Rather than whinge about new webs (AFAICT it's not being created, get over it), I've implemented a TWiki Application using the categories I listed above along with the key features I see being needed on pages like AthensRelease, BeijingRelease, CairoRelease. (These pages mention that they'd like to track features in a more interesting way. It's not perfect - but it's a stab at a more generic version of the Codev web whilst trying to "fix" percieved issues.

I've also had a go at working around the scheduled for/implemented in problems - using TWiki Variables as indicated above.

Based on this the "simpler" version is too complex, Suggestion for even simpler

The version I've put up publicly I haven't populated things in. On a private version based on using more than one webform I have. And based on this I think the approach that we should take is this:

• VERY simple page classification for new pages:
  • SystemDeployment, RelatedProduct, AddOnProduct, DocumentationPage, FeatureDevelopment, BugReport, NonSupportQuestion
  • InterestedParties - Initially includes just the page creator. Other people can add themselves. (This helps IME with conversation tracking having played around with this for a few weeks)
• And that's it.

Then when pages advance to a different stage they change webform to a more interesting form:

• FeatureDevelopmentForm
• BugReportForm
• DocumentationDevelopmentForm

Which contain the specific elements of the uber-classification form. Anything else is way to complicated IMO. Clearly you can have people dive right in with these other classifications, but being confronted with the full thing is pretty nasty for someone new. (We do want fresh blood yes?) Also it adds unecessary structure too early.

Where can I see it ?

The application I created can be found at:

• http://www.yeoldeclue.com/twiki/bin/view/Openproject/HomePage
  • The name is based on the idea it's designed for opensource projects.
• It is configured here: http://www.yeoldeclue.com/twiki/bin/view/Openproject/WebPreferences
• It's uber-webform is here: http://www.yeoldeclue.com/twiki/bin/view/Openproject/ProjectDevForm This is pretty nasty. It's definitely the "all you can eat" version of this though. The leaner, meaner features only version is much nicer/friendlier, hence the comments above.
• It contains a "fake" mini-project which has nominally had 2 releases based on an AardvarkBase and BushBabyBase
• It needs some fake content to show what it can and can't do. I'll add some this evening (probably).

It divides a project into development code bases, and releases issues. This means that features are targetted for a code base (current, base), but implemented in releases. These are named after the current code base until they're released.

Automating TWiki Release Scheduled Features tables

It uses searches to maintain feature tables like we have in CairoRelease by tracking the same 3 percentages. Currently this uses the FORMFIELD retrieving stuff, but this is also a prime candidate for using Crawford's excellent plugin. (I thought about killing 4 birds with 1 stone - improve Codev tracking, find an application for Crawford's app, create a complete example TWiki application, and build something I might need soon anyway for something non-TWiki) This is actually a rather non-trivial application.

As noted above having built this I really think having a simpler first/default form, which MUST be specialised to be noticed for feature or bug development would be a good thing. My 2 cents worth. When I've morphed the version above into something slightly friendlier I'll be making the tar ball available as a TWiki application.

-- MichaelSparks - 16 Jul 2003

Updated with a better set of simple forms

I've overhauled the application I detail above so that rather than using one big form it takes topics through stages. Initially they have the most basic form in use:

• BasicForm - this gives brush strokes classification.
• DocumentationForm - this allows the development and tracking of key documentation type pages. These can range from definition topics to having owners for feature documentation status tracking, and includes keywords, along with an indication of the "Level" of reader the page is targeted at. (Beginner, Advanced, Internals, etc)
• BugTrackingForm - this tracks reporting, and fixing of bugs.
• FeatureDevelopmentForm - this tracks new feature brainstorming, development, implementation, patching, and so on.

If a page ends up spanning multiple forms this is managed by the existance of the CoversEverythingForm . At this stage I think we've got something workable:

• In order to edit a page and help collaborate - provide ideas, there is a minmal barrier to entry - just one categorisation - which encourages the use. (One possible problem I see here is DefineTerm type pages)
• In order for development on a topic to happen the web form must be changed to the right one. This act alone should encourage someone to fill in the contents. (It's the social engineering aspect that helps here- if you don't fill in the parts, it won't get worked on - and likewise can be filtered out)
• Bugs can have their specific form filled in from the BugReport topic - thus ensuring they get used correctly.

A nice result though is that the tables for FeatureTracking which are like those on pages like CairoRelease/etc can reflect activity on those pages since they're searches rather than static. Also pages that have had features checked in can be automatically added to a table - which makes it alot easier to see at a glance what features have/haven't been added into a new release.

Thoughts?

-- MichaelSparks - 16 Jul 2003

Modified Version Live For Tracking a TWiki Fork

I've customised a version of this tool for tracking my personal twiki fork (that I've been running for about 3 years now). Currently I'm just tracking features I'd like to see, and I'm prepared to implement. (Or have implemented) This is live at http://owiki.org/ and since it has read-only (code free) mirroring built in (as I normally have), so a quicker to access version is also at http://www.thwackety.plus.com/owiki/. (The latter is the preferred URL for access since it should be significantly quicker - at the expense of lagging behind the primary site)

As a side note, I'm preparing to release my tree for those who wish to move their TWiki's forward faster, but in a potentially less stable way - I'm waiting to hear the CoreTeam's opinions on the problems with the license.txt file. If I don't get a response within a reasonable period (people disagree on that), I'll just make the release derive from the CVS tree instead. (Which would allow me to comply with the terms of the GPL)

On a side note, PeterMasiar noted that AARDVARK is awkward to spell for non-english spellers so that's now ANTELOPE, and due to the actions of certain world leaders recently, the B release name was pointed out as having political considerations. Hence the B release is now BADGER. Despite protestations at CAT being too short, I'm keeping that as CAT - they're my favourite animal.

-- MichaelSparks - 17 Aug 2003

See my view on a fork in AppealToCodevCommunityByCoreTeam.

-- PeterThoeny - 17 Aug 2003

Micheal I have no feedback right now, and won't for a couple of months, but I fully intend to try this out. One of my biggest complaints with twiki forms is that they essentially force users into database servants, a near universally despised role. I've not voiced any concerns about this till now because I recognise their utility and have not been able to imagine an alternative.

-- MattWilkie - 18 Aug 2003

I'm glad to see the addition of the progress fields on webform. How about the same for patches?

-- MartinCleaver - 15 Dec 2003

I'm not sure what you mean... don't patch topics use the same WebForm?

on the other hand, the Nice to Have list on Cairo will be more difficult. I hope to use a voting system similar to the one Peter is proposing for PluginRateing to rate a features importance

-- SvenDowideit - 16 Dec 2003

How about counting the number of InterestedParties?

-- SamHasler - 16 Dec 2003

I got fed up waiting for something to happen and implemented traditional categories on Codev. See CategoryCategory. If anyone doesn't like it - well, this is a wiki, after all......

-- CrawfordCurrie - 29 Feb 2004

well done, good decision, and cool thankyou

of course, i'll be jealous of your WebStatistics rating

-- SvenDowideit - 29 Feb 2004

Oh bloody hell, I hadn't thought of that - I'll use up my quota if I'm not careful (nobody is allowed to be top except Peter! )

-- CrawfordCurrie - 29 Feb 2004

Gosh. Autonomy! Very wiki-like but so unheard of here! Wonderful!

Thanks Crawford.

-- MartinCleaver - 29 Feb 2004