Igor's Blog

"... no matter what they tell you, it's always a people problem!"

Monday, February 06, 2006

Are there bugs in agile project?

"The main aim of XP is to lower the cost of change."

In many agile projects, one of the most difficult questions is how to deal with defects when they weren’t supposed to exist at all.

Different types of defects...

There are two broad categories of defects - implementation defects and requirements defects:

The best we can do...

Agile development practices are sometimes thought of as an undisciplined approach to software development, lacking such things as effective defect management. On the contrary, agile development does much to reduce the incidence of defects in the first place:

Should we even bother...

Since we apply very rigorously all practices listed above, we expect that very few defects of any type are introduced and the system is in relatively defect-free state at any given time.

Not so fast...

It turns out that most of the time we still end up with a lot of defects at the end of a project (mostly requirements defects) and still spend time and money doing the not so much exciting work of fixing bugs. Finally, we are only humans and we can make mistakes.

So, how do we deal with resolving defects in agile team in the most effective way?

In theory ...

Even the software engineering books very clearly state that "the earlier within the software life cycle a problem is discovered, the cheaper it is to fix". Also, fixing a defect in a much later stage of the project could be a prohibitingly expensive thing to do. This is definitely proven on many occasions to be true.

The strange thing is the fact that not too much has been written about that problem in agile literature. One of the most essential agile principles, however, is that an application delivered by the end of every iteration should be absolutely functional and ready to be used by the customers (even with very limited functionality). Iterative development is perhaps the single most important vehicle for managing requirements defects and in my opinion this should exclude the existence of any them.

In practice ...?

But how many agile projects we know that end up having bug trucking systems with hundreds of bugs there waiting to be fixed. So, we finish the story cards for an iteration on time, we clap hands for one more good iteration where we deliver the most business value to the customer but we don’t bother too much dealing with bugs.

If we go back to the statement above that the defects are getting much more expensive to be fixed in latter iterations or phases of the project it could turn out that we probably didn’t deliver as much business value to the customer as we would’ve liked.

Actually, most of the bugs are left for so called "bug fixing" or "application readiness" iteration(s) at the end of the project. This represents huge risk for a project, since we haven’t made any, even rough estimation for how much time it would take for these defects to be fixed. In reality, we would not be able to do any estimation since we don’t know how many more bugs will be discovered. More importantly, the time and the complexity for fixing current defects will increase in the time to the end of the project.

Is this the right way...?

Of course, there is politics involved here too. Everybody wants to present nice graphs and charts in front of the customer but how is this better than using the waterfall way of thinking?

So, what now...?

Since agile are not some well established processes (thankfully) but rather helpful guidelines, there is no one way to deal with this issue. Every agile team should find its own way to do things through constant retrospectives. Based on my experience while working in agile teams, there are two main assumptions that tend to lead to the bug problem in an agile project:
  1. The assumption that every software development project will indisputably need bug trucking system.

  2. The bugs are not prioritized together with the story cards in relation to the business value they represent.

Is this even open to debate...????

The first assumption is very similar to the assumption that we need a relational database in every project we are about to start. Probably, most of the time we do need some kind of database, but we should not just assume and implement some kind of infrastructure that is not driven by business needs (this is how we end up with so call "technical cards" that can not be justify with any business value for the customer but I am leaving this for other time).

Did we overlook communication...?

Since every tool imposes some kind of process to be followed in some degree, an agile team should be particularly alert when picking tools to work with. Overall, an agile team should choose "people and communication over tools and processes". Sure enough, the bug trucking tools do a good job of keeping record of what kind of defects has been discover and much more information that we will ever need.

However, we should determine whether we loose the communication within the team by using such tools. What happens usually in practice is that a BA, QA or a user discovers a presumable defect in the system and enters it in a bug tracking database. Much later, probably at the end of the project, a developer opens the same bug to work on. By that time, all of the communication and most of the information about this bug is lost. The reasons could be many:

As a whole, later defect detection and repair is much more expensive, which is totally against one of the main aims of Agile development to lower the cost of a project.

Defects and feature request are...

Recent innovative software development methodology is Defect Driven Design (D3, pronounced "D Cubed"). The idea is to remove the two major phases in a project development phase (as some call it - defect creation) and maintenance phase (also called defect removal) and have only one phase called probably defect implementation phase. You can read more about this idea but the main point was that there is no distinction between a defect and a new feature request, story card, enhancement or whatever you want to call it.

What provides more business value...?

Even though D3 is still very open to debate, the idea that defects should not be separated from and prioritized differently than story cards is not easily ignored.

Usually customers don’t have problems prioritizing new functionality in regard to importance and business value for them. However, we would not see the same level of confidence when we ask customers, whether they prefer to have new functionality added or some previously prioritized and not currently working functionality fixed. Of course, the answer strongly depends on case by case basis but that’s why, we should not make assumption so that to eliminate defects from the prioritization process and treat them as something different than story cards.

Is there another way...?

It is not a big surprised the fact that there is not one efficient way to deal with defects in different agile teams. The same should be true for the assumption that bug tracking database is the best solution for every software project. We can have or try many possible scenarios:

How would we know it is better...?

Iterative development and team retrospectives after every iteration are perhaps the single most important vehicle for managing defects. Through retrospective and fast feedback we can determine how effectively we manage defects and whether we detect and fix them as early as possible. The goal is very clear and even more easily confirmable - we should not have bugs fixing iterations at the end of an agile project.

What actually I wanted to say...

The whole thing was to reassure myself and convince at least one more person that:
A very good addition to this is "... on features vs task post":

Instead of recognizing that defects must be prioritized and planned as stories, as Jon's post suggests, defects are still considered "exceptions" as per traditional thinking about software development. There is a tendency to avoid treating them as real work items.

|| Igor, Monday, February 06, 2006


defects exists in many agile projects because tests aren't engaged properly on the team.
Anonymous Anonymous, at February 07, 2006 1:53 PM  
Right On!
Anonymous Anonymous, at February 10, 2006 3:32 PM  
That is a total load. You can have 100% test coverage, and still have missed some edge case that no one anticipated. I have seen too many agile projects smugly point to their test modules and coverage charts and assert "We have no bugs" only to point out a number of holes in the app, and not necessarily because the quality of tests was off. It is a false sense of security thing. Get over it, accept that bugs happen, and deal with them as effectively and disruption-free a manner as possible.
Blogger bonevich, at February 12, 2006 2:39 PM  
“That is a total load. You can have 100% test coverage, and still have missed some edge case that no one anticipated.”

Completely agree. However, I not saying that a project could be bug free but not to use bug tracking tool for keeping truck of bugs. Instead we can create a story cards for these bugs and prioritize them as the other story cards as soon as possible.
Blogger Igor, at February 12, 2006 8:35 PM  
That is exactly what we do. Throw all issues into the pot and let the business decide which stories they want done each iteration.
Blogger bonevich, at February 13, 2006 8:48 AM  
bonevich: Ms Onymous mentioned tests not "properly engaged," she didn't say anything about coverage statistics. Obviously coverage statistics can be misleading. I understand "properly engaged" to mean the tests are well thought out to expose potential errors. No one would claim this can possibly eliminate every potential bug, of course.

I also agree that all issues go into the pot, as you say.
Anonymous Anonymous, at February 14, 2006 12:05 PM  
Thoughts are well discribed, good going...
Anonymous Anonymous, at October 06, 2006 6:14 AM  
In agile development the business analyst, product owner, developers, and qa never spend or have enough time to groom an item down to its most granular details. For the most part an item is granulated down enough where each product backlog item can have a realistic estimate, which is always underestimated due to lack of details and further refinements. The items that get developed during the sprint have conditions of acceptance and requirements tweaked, updated, redefined, clarified, until at the end of the sprint cycle the developers have done their best job in interpreting the requirements of the items and qa takes their best stab at putting together the test cases that can be executed. The test cases never encompass all of the requirements since they are in a constant state of change until something is delivered that needs to be analyzed. Then there are always changes there and both developers and qa struggle with getting something right. This results in lots of bugs at the end.
Anonymous Anonymous, at November 28, 2009 12:44 AM  

Add a comment