Igor's Blog

(Updated)
"The main aim of XP is to lower the cost of change."

In many agile projects, one of the most difficult questions is how to deal with defects when they weren’t supposed to exist at all.

Different types of defects...

There are two broad categories of defects - implementation defects and requirements defects:

• implementation defects refer to defects in architecture, design, coding, installation, or any other aspect of the technical implementation of a software development project. Implementation defects are a major source of project trouble in traditional methodologies but no so much in agile practices.


• requirements defects include all types of requirements, functional or story card specifications, acceptance testing (probably FIT) and any other means to define what the software is supposed to do and, from a functional perspective, how it is supposed to do it. Requirements defects are very relevant for "building the right software" and as such they are the most disastrous and costly.

The best we can do...

Agile development practices are sometimes thought of as an undisciplined approach to software development, lacking such things as effective defect management. On the contrary, agile development does much to reduce the incidence of defects in the first place:

• we have all types of automated tests - unit, integration, regression, functional, acceptance and so on.
• we do TDD to have very fast feedback and to ensure that the code is thoroughly tested.
• when we pair we have the benefit of more than one person watching the quality of the code.
• we do continues integration in order to uncovers integration issues early.
• we have short iteration cycles in order to verify that everything built until this point is fully functional according to the story cards and reviewed by the customers.
• we put the customers (users) in the center of the project to ensure adequate client involvement and proper set of project priorities.
• we communicate with the customers (users) constantly to ensure we evolve our understanding of the business domain, challenge assumptions, and make informed choices and decisions.

Should we even bother...

Since we apply very rigorously all practices listed above, we expect that very few defects of any type are introduced and the system is in relatively defect-free state at any given time.

Not so fast...

It turns out that most of the time we still end up with a lot of defects at the end of a project (mostly requirements defects) and still spend time and money doing the not so much exciting work of fixing bugs. Finally, we are only humans and we can make mistakes.

So, how do we deal with resolving defects in agile team in the most effective way?

In theory ...

Even the software engineering books very clearly state that "the earlier within the software life cycle a problem is discovered, the cheaper it is to fix". Also, fixing a defect in a much later stage of the project could be a prohibitingly expensive thing to do. This is definitely proven on many occasions to be true.

The strange thing is the fact that not too much has been written about that problem in agile literature. One of the most essential agile principles, however, is that an application delivered by the end of every iteration should be absolutely functional and ready to be used by the customers (even with very limited functionality). Iterative development is perhaps the single most important vehicle for managing requirements defects and in my opinion this should exclude the existence of any them.

In practice ...?

But how many agile projects we know that end up having bug trucking systems with hundreds of bugs there waiting to be fixed. So, we finish the story cards for an iteration on time, we clap hands for one more good iteration where we deliver the most business value to the customer but we don’t bother too much dealing with bugs.

If we go back to the statement above that the defects are getting much more expensive to be fixed in latter iterations or phases of the project it could turn out that we probably didn’t deliver as much business value to the customer as we would’ve liked.

Actually, most of the bugs are left for so called "bug fixing" or "application readiness" iteration(s) at the end of the project. This represents huge risk for a project, since we haven’t made any, even rough estimation for how much time it would take for these defects to be fixed. In reality, we would not be able to do any estimation since we don’t know how many more bugs will be discovered. More importantly, the time and the complexity for fixing current defects will increase in the time to the end of the project.

Is this the right way...?

Of course, there is politics involved here too. Everybody wants to present nice graphs and charts in front of the customer but how is this better than using the waterfall way of thinking?

So, what now...?

Since agile are not some well established processes (thankfully) but rather helpful guidelines, there is no one way to deal with this issue. Every agile team should find its own way to do things through constant retrospectives. Based on my experience while working in agile teams, there are two main assumptions that tend to lead to the bug problem in an agile project:

1. The assumption that every software development project will indisputably need bug trucking system.


2. The bugs are not prioritized together with the story cards in relation to the business value they represent.

Is this even open to debate...????

The first assumption is very similar to the assumption that we need a relational database in every project we are about to start. Probably, most of the time we do need some kind of database, but we should not just assume and implement some kind of infrastructure that is not driven by business needs (this is how we end up with so call "technical cards" that can not be justify with any business value for the customer but I am leaving this for other time).

Did we overlook communication...?

Since every tool imposes some kind of process to be followed in some degree, an agile team should be particularly alert when picking tools to work with. Overall, an agile team should choose "people and communication over tools and processes". Sure enough, the bug trucking tools do a good job of keeping record of what kind of defects has been discover and much more information that we will ever need.

However, we should determine whether we loose the communication within the team by using such tools. What happens usually in practice is that a BA, QA or a user discovers a presumable defect in the system and enters it in a bug tracking database. Much later, probably at the end of the project, a developer opens the same bug to work on. By that time, all of the communication and most of the information about this bug is lost. The reasons could be many:

• the most important case is when we have requirements defects where a lot of functionality is built on top of wrong assumptions. This should not be so much severe in well designed OO systems but still it is going to be much, much more expensive than fixing a defect around the time when it was discovered.
• most if not all of the bug trucking tools require owners assign to a defect. This is very close to code ownership and doesn’t stimulate the other team members to participate in the knowledge sharing while resolving a particular defect.
  
• it can’t be reproduced since many times the cases are very data specific.
• many people that had worked on a particular functionality have left the project already.
• developers and BAs have a clearer picture of what is going on with the system when a story card is implemented but they have hard time recall any issues latter on.

As a whole, later defect detection and repair is much more expensive, which is totally against one of the main aims of Agile development to lower the cost of a project.

Defects and feature request are...

Recent innovative software development methodology is Defect Driven Design (D3, pronounced "D Cubed"). The idea is to remove the two major phases in a project development phase (as some call it - defect creation) and maintenance phase (also called defect removal) and have only one phase called probably defect implementation phase. You can read more about this idea but the main point was that there is no distinction between a defect and a new feature request, story card, enhancement or whatever you want to call it.

What provides more business value...?

Even though D3 is still very open to debate, the idea that defects should not be separated from and prioritized differently than story cards is not easily ignored.

Usually customers don’t have problems prioritizing new functionality in regard to importance and business value for them. However, we would not see the same level of confidence when we ask customers, whether they prefer to have new functionality added or some previously prioritized and not currently working functionality fixed. Of course, the answer strongly depends on case by case basis but that’s why, we should not make assumption so that to eliminate defects from the prioritization process and treat them as something different than story cards.

Is there another way...?

It is not a big surprised the fact that there is not one efficient way to deal with defects in different agile teams. The same should be true for the assumption that bug tracking database is the best solution for every software project. We can have or try many possible scenarios:

• some kind of white board (either real or wiki based) where everybody can see current development of defects, the progress and all related communications.
• customers and anybody else involved in acceptance testing can use some kind of recording test tools when they do manual testing. Firefox has very simple and user friendly plugging for Selenium test for example. With such a tool, a customer can very easily record any defect and then send the resulted failing test file. The file could be included immediately in the build and thus fixed as soon as possible. No time lost for reproducing errors or misunderstanding or some kind of blaming.
• creating story cards for defects that are prioritized, estimated and accepted as any other story card is very appealing scenario. This will ensure that all defects are analyzed, discussed, prioritized and fixed in proper manner and time. There could be slight variations like creating story cards with letters or numbers in the name. Example, Search For User, Search for User- b or Search for User When something happens and so on.

How would we know it is better...?

Iterative development and team retrospectives after every iteration are perhaps the single most important vehicle for managing defects. Through retrospective and fast feedback we can determine how effectively we manage defects and whether we detect and fix them as early as possible. The goal is very clear and even more easily confirmable - we should not have bugs fixing iterations at the end of an agile project.

What actually I wanted to say...

The whole thing was to reassure myself and convince at least one more person that:

• many times we don’t need buck trucking databases since these kinds of tools almost always expose some process to be followed and the huge benefits of communication in an agile teams is lost.
• defects are actually missing or not properly working features of the system and as such we should not make distinction between story cards and defects.

Update:
A very good addition to this is "... on features vs task post":

Instead of recognizing that defects must be prioritized and planned as stories, as Jon's post suggests, defects are still considered "exceptions" as per traditional thinking about software development. There is a tendency to avoid treating them as real work items.