TWiki has lost its Customer Focus

I joined the TWiki project at the time Cairo was released.

Cairo was a relatively stable program. It had a few bugs which you could fix with a couple of patches but overall it was stable.

TWiki 4.0 (Dakar) was a major rewrite and also a release with quite many bugs. But the community was fast at reacting with 4.0.1, 4.0.2, 4.0.3, 4.0.4.

TWiki 4.1.0 (Edinburgh) was also followed up by 4.1.1 and 4.1.2.

With 4.2.0 we got again some major code refactorings. And as always bugs surfaced. But this time the commitment to fixing bugs seems at an all time low. And it is not because people do not have time. Codev is glowing with ideas of major refactorings and new features. And plugins are being updated. But the core code is stalled and the list of urgent bugs is growing and I cannot get 4.2.1 to a stable state where I can release it even though the list of urgent bugs is not larger than we could fix it all in a few days.

And lately some of the most active developers have stopped participating at the release meetings meaning that we cannot discuss and help each other with the bug fixing and people's feature proposals for 5.0 are not getting the attention they deserve.

Is there an interest in getting 4.2.1 released?

Is there an interest in serving our users? Or is it the idea that our users should be punished because a couple of people cannot pick up the phone and get some petty little disagreements settled?

Why should new users install TWiki? Why should anyone buy services from any of the companies or consultants associated with TWiki?

Frustrated words from a very disillusioned release manager.

-- Contributor: KennethLavrsen - 29 Apr 2008

Don't be frustrated Kenneth, look at EnginesAsContribs as an example that core isn't stalled at all. But of course you're right that 4.2.1 should be released soon.

-- FranzJosefGigler - 29 Apr 2008

when I released Cairo, there were exactly the same complaints that there were too many re-written bits. it also took several patch releases before there were enough users to report and iron out the bugs. Diference there, is we have a bug tracking system, and very few people that fix bugs they report.

As the -other- release manager, I think you are not taking enough information into account.

-- SvenDowideit - 29 Apr 2008

Franz - yes lots of energy on the EnginesAsContribs. And very little energy on fixing the short list of bugs marked "Urgent" in the bugs web. People are doing the "fun" things only and forgetting the customers that struggle with the current release and the new bugs that were introduced in it.

The issues are

• 4.2.0 is not followed up by a much needed 4.2.1
• Release meeting process is not working when people do not participate

-- KennethLavrsen - 30 Apr 2008

Dirty Washing? Should we really be airing issues like this on the TWiki blog? If I was a manager in the process of checking out TWiki whilst evaluating which wiki to use in my company, this sort of talk would really put me off TWiki since it makes it sound like a dead project. Which of course it is not.

Could we please hang out our dirty washing "inside" TWikiDotOrg? We've got a nice big - albeit slightly cluttered - airing room in the shape of Codev.

-- MichaelCorbett - 12 May 2008

Listening to my customers that are also 'TWiki' Customers, I get the feeling that TWiki has not lost its customer focus. They are thrilled that they can get access to the features made available through the refactorings, and so far, I've been quite busy making their desires into reality.

Over the last months, I've fixed their most urgent bugs in 4.2.0, and the result of that can be seen in the statistics on AllOutStandingItems.

We all need to prioritize, but complaining that other people's priorities are not identical to yours, are not a positive way forward.

-- SvenDowideit - 12 May 2008

Kenneth, you are well known by many examples for fighting for the customer interests, i.e. motorola, reminding us all how important this point of view is. That's good ... in general.

However, you are by far not the only person concerned about customer focus. Still, it seems as if you use the "customer focus" argument less of a way to add value to a discussion but more as a way to express your own unease. That's no good. It blurs the view for the real problems at hand.

Being a release manager does not necessarily mean to become that negative and aggressive. (/me pictures an old blind prior lashing on his monks ).

The unpleasant sideeffect of this is, that you create the impression of two frontiers: on the one side you advocating for customer focus - on the other side the rest of us. This pattern in your words emerges all too often.

However, I must subscribe to your perception of the project being at an all time low tide. Alas, your conclusion TWiki core contributors may have lost customer focus is not correct! And I am not elaborating this at all .

Reality is, that 4.2.1 is not out, and people are in a bad mood to work on it. Why this is so? That's a valid question you ask, Kenneth. I want to know the answer as you do, and want to continue to working on TWiki with friends and in a good spirit.

Think of other possible reasons for the problems we currently have. The one being heading this unfortunate blog entry are quite off, really. Might there be a reason in your own doing? Is this really all the fault of TWiki's core contributors, the few that did not yet fully shy away, not fixing bugs?

-- MichaelDaum - 18 May 2008

I totally agree that it is not the purpose of the blog to post accusations and flames. I understand that a good will and strong believe in the power of the community lies in your post, but it is the action that count. It is no good to complain in public. The public should understand, why TWiki is a poweful software and is run by a fun community.

I am new in this community and I have a very good impression. Do you know, what the only thing is that turns my down: It is all the critique about things not evolving in the right way. I do see a lot of movements. I do not feel, that any participant wants to hurt the community. We are all fighting together against the competing wiki-software-systems. These are the enemies to be flamed.

-- MartinSeibert - 18 May 2008

so move this article into codev?

-- CarloSchulz - 19 May 2008

Kenneth, you know very well what the reasons are for this lack of motivation. There is more to this than 'some petty little disagreements' that can be solved by a simple phone call.

Anyway, i have largely lost interest in these issues, and moved on to other fun projects. Because that is why i initially got involved in TWiki: having a good time. Too much has happened by now, and i've found out that i don't want to work in an environment with any kind of dictatorship. The fun is gone, twiki is not for me.

For now, i'll stand by on a distance, observing how issues are shoved under the carpet with a false smile and some fake amiability, glad not to have to worry about it anymore.

-- KoenMartens - 21 May 2008

Carlo is right, it was time to move this topic to the Codev web, which I did.

Overall good opinions here, although to my taste a few could be more constructive. I am with Kenneth that we have a lack of focus on getting 4.2.1 released, e.g. we do not have enough coders fixing bugs. For a while a key contributor stopped altogether, now we have bug fixing activity again. However, I can understand the frustration of Kenneth and other folks (including me) who see the lack of participation. Personally it is frustrating to see this delay in getting a high quality TWiki product released.

I am also with Martin, we have a lot of momentum in marketing (video contest, twitter, etc) and development (new proposals/implementations such as TWikiStandalone and SupportAccessToArbitraryMetaDataFromSEARCH). Some people are also actively working on recruiting new people, which is very positive. More contributors bring more balance which brings more fun to the project. (welcome ClifKussmaul to the core dev team!)

I am not going to address Koen's accusations on dictatorship, but I'd like to say that it is quite sad to see that statement facing what I am doing for the community, working 12-14 hours a day on TWiki, half of that on twiki.org stuff.

-- PeterThoeny - 21 May 2008

People are able to work at a company even if they don't like the CEO. Because they love the work, the colleagues. And because they believe in the things they are making.

Yes I believe an OS project is like a working place, with its frictions and irritations and competition. Yet the company would not survive if it did not have focus.

The TWiki project has entered a crucial phase. We need to act, and to regain focus.

The number of working hours on the project is not the point. Nor is it the lack of developers working on outstanding bugs.

We need to address the issues that are really important. We need to do better at making sure people are enabled and empowered.

What role does the community play? Who gets the responsibility and the fame? What does a "core team" mean? How do we vote or reject? How can we get better understanding between the commercial entities and the OS project? And how are issues resolved?

These are important matters. Discussion on these are halted. We need a driver on our own process.

-- ArthurClemens - 21 May 2008

I stand by every word I said. The Bugs:ReleaseBlocker has hardly moved since I wrote the blog entry a month ago. We are still not nearer the release of 4.2.1. The focus is on what is fun and on internal power battles and not on getting a release out to our customers.

And now on top blogs are being censored

• I see this differently: I think it is better to handle community problems within the community, hence I believe it is better to discuss the customer focus question in the Codev web where all developers and marketeers hang out. The blog is very public; if I as mister evaluator look at TWiki and 2 other choices I'd stay away from TWiki reading comments like these. (Busy people do not have the time to probe to find out how healthy a community is (we have some miles to go, but overall we are weathering storms.)) -- PeterThoeny - 22 May 2008

-- KennethLavrsen - 21 May 2008

Hard to think of anything positive to say in light of all this negativity.

My users are thrilled that I'm busy working on fixing their issues, and most importantly, using TWiki to develop the Knowledge applications they need. Perhaps you're saying that I'm customer-focused, specifically because I'm working on my customers?

Why, thankyou!

-- SvenDowideit - 22 May 2008

I will become positive when I see the old bugs (some have blocked the 4.2.1 release for 5 months and should have been fixed before we released 4.2.0).

Currently 4.2.0 gives access rights issues when using login IDs and if you are a non-English user you will for sure have problems if you do not use English only in the Wiki.

The sad thing is that the current TWikiRelease04x02 branch already have fixes for 95% of the most severe issues. The urgent bug list is not that long and some of them are marked urgent because a developer thinks it should be included for a specific purpose. But the real list of bugs that I consider as release blockers are

• Bugs:Item5339 which I have a quick hack fix for
• Bugs:Item5626 problems with non English
• Bugs:Item5443 causes problems for existing TWiki apps
• Bugs:Item5529 prevents use of UTF8 which is the only reasonable way to use language such as Chinese, Japanese, Korean etc.
• Bugs:Item5638 new item that will cause trouble for a lot of people if they install on a machine with perl 5.10
• Bugs:Item5566 prevents use of UTF8
• Bugs:Item5602 which in reality is a serious security issue
• Bugs:Item5118 which causes trouble for corporate installations. I am personally fixing issues related to this at least once a week
• Bugs:Item5514 which makes a bad impression as all users experience errors reported by the browser when they edit topics. Not a functional defect but leaves an impression of TWiki being broken.
• Bugs:Item5528 again prevents use of UTF8

When this community have been close to a release we were able to - with joined forces - close 2-5 bugs per day. If we focus we can have 4.2.1 released a week from now and make a lot of customers happy.

Sven you have 2 or 3 bugs that are the result of the user mapping changes introduced in 4.2.0 and that several people have said they would try to fix including TWiki.net people. But it seems you are the only one that understands how the code it supposed to work.

Can't you spend a day or two getting these out of the way please?

With the I18N (UTF8) bugs you cannot say the bugs are related to changes made by a specific person and we - as a team - clearly do not know enough about UTF8 and perl. Can we somehow team up and help each other on these. Maybe with a UTF8 code session on IRC where 5 or 6 of us spend a few hours in a weekend to analyse and suggest things. I have personally spent many hours on it but often get stuck on some perl technical detail which I know most of you can help with. For example "how do I identify and poor-man-debug if Perl thinks a specific variable is in UTF8 format?"

What makes me frustrated about the whole deal is that we are so close to being at the goal line.

Please help getting those few bugs fixed so our customers can get a much more stable 4.2.1. And then we can play with 5.0 features as much as we like as well as getting the open governance issues resolved.

-- KennethLavrsen - 22 May 2008

I have to agree with Sven here. I took a decision after the summit in February to back away from core dev martyrdom and focus instead on specific customer requirements. Since then I have had lots of positive feedback, a lot less sniping from within the TWiki community, interesting and fun people and projects to work with, and yes, life with TWiki is slowly starting to be fun again. To the extent that I have even worked on some features (international encodings) that are of no value whatsoever to any of my clients.

As Arthur so clearly says, this is not about hours worked or headcount. It's about creating an environment where people feel so happy about contributing that these things just happen.

-- CrawfordCurrie - 22 May 2008

But how do we get 4.2.1 released if we only do the fun parts?

I spend quite many hours on TWiki as a contributor. And I think you must agree that I take the most boring part of the work.

My reward has been getting a better and better TWiki. And that is all I ask for.

I need the 4.2.1. 4.2.0 added some fantastic new features (TMCE and Query search) so reverting to 4.1.2 is not the solution for me. But 4.2.0 also created some trouble for me and my users.

It is not fun for me to contribute to this project if my needs and problems get dismissed. I do not personally need all 10 urgent bugs fixed for my needs. My problems relate to 2 or 3 of them. And I have tried - and spent many hours - to solve for example the 5118 myself. But the code is just too complicated for me. I can spend 1000 hours testing and verifying fixes but I do not understand the code to be able to program the solution.

Please please help getting these bugs fixed so I can get my angry users calmed down. I am in a constant fight internally with people that are against using TWiki and it is the bugs are their big weapon against me.

-- KennethLavrsen - 22 May 2008

Just a pragmatic question: Does the i18n/UTF-8 bugs exist on TWiki 4.1.2 or where introduced on TWiki 4.2.0? Those that come from 4.1.2 should not stop us from releasing 4.2.1.

More so, as we're talking about "dot" releases, I think that the best thing we can do is to release periodically (once a month?) with whatever bugs where fixed. In my case, and I bet it's not an uncommon case, as a Customer I only care about some of the bugs in your list (and some other what are already fixed on SVN), thus I would be more than happy to receive a path release that fixes them.

Patch releases should not be treated the same as Major or Minor releases, they should have only one blocking criteria: It works at least as good as all the versions in that family (meaning, no regressions).

At least, this would give the outside viewer the (true) impression that we're moving forward, not that we're in a staled state (which is not true)

-- RafaelAlvarez - 22 May 2008

I totally agree with you, Rafael.

-- MichaelDaum - 22 May 2008

With respect to UTF-8. UTF-8 as such has never been claimed to be supported. It has been regarded as experimental. But we have had a number of Chinese speaking people report problems with the new Wysiwyg editor. Making just a reasonable UTF-8 support enables huge language areas in this world, to start using TWiki. So there is a heck of a good marketing drive behind getting the UTF8 working better. It is a bit sad having to say to users that used TWiki well in Chinese with classical edit that they cannot use Wysiwyg. And we have made significant progress. As far as I know the 4.2.1 code base enables writing Chinese now. Only severe issues is some bugs related to searching in UTF8 mode and verbatim blocks. So we are dammed near having a useful (not perfect) UTF8 support now.

It takes a couple of weeks with 4-5 hour working days + weekends to get a release ready. I don't have that kind of time. And I have not felt motivated to release 4.2.1 as it has been until now because the most important customer to me - me - does not get his problems resolved. Do not underestimate the effort needed by us release managers (Sven and I) to get a release staged and ready.

An unattended build that creates a nightly or weekly zip is another story but who will put something like that in production?

-- KennethLavrsen - 22 May 2008

Hi Kenneth, I see in your thorough involvement in discussing this issue, that you truly want to solve problems and try to bring TWiki not just one but multiple steps ahead. I appreciate that and I agree, that every successful project is in need of critiques and people who analyze things and point to the weak points. But if we are talking about the people working for me, I request them to give me an idea of what they believe, what a solution should be. It is difficult for the whole industry to get resources from skilled programmers. I for example are normally in sales and consulting in our company. But there is little to sell at the moment, if we are talking about programming ressources. We should not be surprised if TWiki is lacking programming ressources. A lot of people in the market do.

I agree with Peter to move the topic to codev. This is not censorship but marketing. We want to show off the best of TWiki to the public. On that certain thing, I would agree with Koen, that it is better to shove these thing under the carpet.

Koen: But as you see, there are a lot of people who care under that carpet. Nobody should post a flame like you did above. It is destructive and accusing. I would hope, that you - like all others do - try to better things by analyzing them and giving useful suggestions.

Kenneth: //SEIBERT/MEDIA has more than 30 perl-programmers. If we had resources. How should they be used? I am talking about a general lack, that we have in recruiting people. If I am interested in TWiki, how can i contribute. I know, that there are some documents on that. But we should revise them and make sure, that the path is especially easy to follow for programmers. How can we lead new programmers to their first contribution?
This is also largely an issue of usability. Ask Carlo!

By the way: my personal opinion is, that more and more commercial activity - like Peter's - can do a lot of good for the community.

All this stuff should be part of the next summit. I will put it to the agenda and call it "*Discuss solutions for political challenges*". My hope is, that every participant has thought about what he thinks, will lead to a solution until then. We need no more discussion about the problems. That is well documented.

Who knows, what those, that are involved can do. There seem to be resources from non-programmers available. Use them!

-- MartinSeibert - 22 May 2008

I fully agree with Rafael.

No offense, but on the Chinese part. It's not in their culture to use a wiki. I wouldn't put so much hope anytime soon.

Martin, I can agree with you about commercial activity. But I must say, I'm still waiting for it to happen soon. I think any more discussion will just be hot air. As far as my memory serves me right, lots of these core discussions were discussed time and time again, yet little improvements were made. It'd be good to root out the cause of that rather than what we see on this surface.

-- KwangErnLiew - 22 May 2008

_An unattended build that creates a nightly or weekly zip is another story but who will put something like that in production? _

HUH? I have had a nightly build of the 4.2 branch running since September 2007! Setting it up to make a 'real dot release' every month is trvial, and if there is a desire for this, I will happily 'make it so'.

my nightly 4.2 branch build output

(mmm, it looks like the VM had a problem since my disk corruption of 12-May, I'll get it running in the next day or so)

-- SvenDowideit - 22 May 2008

Rafael, Michael and Kwang. Who is going to do the work making a patch release each month? Who is going to run the needed tests to ensure that a particular snapshot is release quality? Who can throw in round 50-70 working hours every month making these releases. I am not able to. I would love to see a monthly release. But none of us can find that kind of time. We can make a patch releases 3-4 times per year.

Martin. If we have not solved the governance issues BEFORE the next Summit then I propose we cancel it. I am not prepared to experience a repetition of what went on at the last Summit. That was a BAD experience. I want to discuss code and features and marketing and future among friends that shares the same passion.

Sven. It would be great to get the nightly builds working again. And I would let them build even if unit tests fail. But these nightly builds can be terribly broken one day and great the other day. Noone can know until they have installed it. I doubt many customers will want to install untested versions in a production site. But the nightly builds are great for verifying fixes and similar evaluations.

If someone wants an "untested release" then I have one for you anytime. I just built one now. One command and you have a zip 5 minutes later. That is no issue.

http://www.lavrsen.dk/misc/TWiki-20080523.tgz

-- KennethLavrsen - 23 May 2008

Kenneth, I only said one month as a suggestion. It could be one, two, three, four months... All I'm just saying is that we should timebox the patch release, and if at the end of that period we have something stable with some bugfixes, release the darn thing! It really doesn't matter if all the bugs were squashed, as each release will be better than the previous one.

My main point is that patch releases should not be treated the same as Major and Minor releases: They should be released "periodically" (or as often as it makes sense) as long as there are no regressions and no major bug is introduced.

Otherwise, we risk giving the impression that development has "stalled", which has not: a lot of fixes are in SVN.

If the blocking point is that testing TWiki takes 50-70 working hours, then we should find a way to tackle that (unit testing is not enough, something like Selenium may be needed). The good thing about a path release is that the "UI" does not change much, so automated testing using robots is feasible.

-- RafaelAlvarez - 23 May 2008

Read about half of that topic. Overall it seems a bit of an overreaction. TWiki did not loose it's customer focus. We have delays in a software release. So what!?! Seems to me like a very common problem in software projects. Shit happens! You could say TWiki lost it's customer focus if we started implementing coffee making software which we didn't obviously. Maybe we should rename that topic to something like WeMustRelease4Dot2Dot1Asap.

If it takes so long for core staff to fix issues in 4.2.0 it's maybe because they are not that critical and 4.2.0 is not that bad (Just a guess).

Personally I never use dot zero releases in production and I'm kind of waiting on 4.2.1 to do that. So from a marketing point of view sitting on dot zero release for so long is not that good.

Another thought: seems to me like focus is not really the strength of OS projects but rather their stamina.

I'm not saying we can't improve our focus on releases though. Anyway I open my big mouth but I'm not even a core developer nor am I planning to be in the foreseeable future.

-- StephaneLenclud - 23 May 2008

It has always been my view that installing and testing both beta versions and the .0 versions and reporting bugs is as vital a contribution as coding. But based on my experience from this spring I will probably take the same approach as you Stephane in future. Just like I learned that blogging is not a media for me. At least not here in twiki.org where the Blog web now has become moderated and censored by the Marketing league.

-- KennethLavrsen - 23 May 2008

Never tried the betas and don't even remember seeing them being published. However I did set-up and use quite a few 4.2.0 for evaluation and plug-in development purposes. I'm just a bit reluctant to deploy 4.2.0 in production environment cause it's a dot zero (just my superstition) and it seems to me that I will need to spend significant time and effort in gathering various patches for core and plug-ins (ldap, NatSkin ...). Then again I have very limited time I can dedicate to TWiki development and I'm more than happy with my 4.1.2 installations I use in production environment.

-- StephaneLenclud - 23 May 2008

I think Rafael as a point there test and QA automation is the key to regular stable patch releases. The release process (making and publishing ZIP and TGZ) should be fairly easy to automate for any Perl capable TWiki developer. I quite like JMeter for integration test automation.

Funny enough in my company TWiki is central to our Symbian software release process. We automate build, release and publish software using TWiki as a CruiseControl if you want.

-- StephaneLenclud - 23 May 2008

I do not care if this topic is called "TWiki has lost its Customer Focus" or "release 4.2.1 asap," it means the same for the user base.

Kenneth, I respectfully disagree about the "censorship". Please compare the number of posts while this topic was in the Blog web (0.4 posts/day) to the number of posts we have here (10 posts/day). Contributors naturally hang out more in the Codev web.

Rafael brings up a good point: Release more often, even with outstanding bugs. With our current process we have an "urgent" bug state, which is a "release blocker". How about distinguishing between major/minor release and patch release? For the former, an urgent bug needs to be a release blocker. For patch releases may be it makes sense to have two types of urgent bugs: Showstoppers bugs that must be fixed asap, and bugs that need to be fixed but should not hold up a patch release.

-- PeterThoeny - 23 May 2008

Thanks for bringing back the focus on the focus discussion.

-- MichaelDaum - 23 May 2008

Kenneth said: If we have not solved the governance issues BEFORE the next Summit then I propose we cancel it. As I would like the German Summit to be a success, I would like to solve the governance-issues be heating up the Foundation-topic. Would you all help in giving your votes here: TWikiFoundation

-- MartinSeibert - 24 May 2008

I agree with Kenneth about the need to address governance before the next summit. In terms of getting the community on a firm foundation, no amount of phone calls or tweaking of release processes can take the place of establishing a framework for transparent and accountable TWikiGovernance. There was a vitally important promise made at the last summit regarding reformation of the core team that remains unfulfilled. In terms of leadership, doing what we say we'll do and addressing these foundation issues is what counts — not doing minor editorial clean-up on twiki.org.

-- LynnwoodBrown - 27 May 2008

(This article is move from Blog.2008-04-29-twiki-has-lost-its-customer-focus)