Should we consider a story “done” and then handle the bugs separately or should we fix the bugs as we spot them?
To answer this question one should consider many facts:
- Size and complexity of the product
Simpler software products will always have less use cases and therefore less potential for obscure bugs.
- Size of the iteration
- Scrum team setup
In teams consisting of pure roles with dedicated QA and dedicated development one has to document bugs. This is where the best QA professionals think outside of the DoD and will always find bugs from the most obscure use cases. Teams without a dedicated QA force often test only upon DoD according to the common sense.
- Stakeholder outlook
If the stakeholders consist primarily of experienced software professionals, they may actually know that a documented low-priority bug means that the QA is doing a thorough job.
On the other hand, a story with bugs is not a shippable story.
Bugs are always found during the sprint: they are kind of expected and should be fixed or made acceptable as defined by the acceptance tests. Finding bugs and errors has to be a part of the sprint process. During the sprint, all sprint backlog items are validated for acceptability by the QA/PO and those that do not pass by the end of the sprint are put back into the backlog.
Therefore, the story does not get included in the release of that particular sprint if it is not acceptable.
Due to this the team’s velocity lowers.
While there is certainly a great value in fixing bugs as they were found, one should be careful to not create a QA-vs-Devs race by the end of the sprint, which usually looks like bouncing a story back and forth.
At the same time, one should also be careful not to run into a situation when the team doesn’t document bugs because they are minor and would prevent the story from being done. All bugs should be filed when found. It’s actually the decision whether fix them or not that should be taken to the triage part of the scrum ceremonies.
Cost-Benefit Analysis Solution
What works well is to define an abstract defect severity (using some scale of values e.g. story points or T-Shirt sizes) and agree (as part of the definition of done) which level of severity should prevent a story from being finished. Any bug that with severity less than a certain critical value could be postponed to the next sprint. Other bugs should be fixed on the spot.
Conclusion (Rule of Thumb)
- If it’s faster to fix a bug rather than document it — fix it.
- If the severity is more than critical — fix it.
- If there is not enough time to fix it within the current sprint — ask the PO, but propose your solution.