Death by a thousand bug reports – Part 2: Rescue of the drowning

avoid being flooded by software bug reports

I assume you’ve read the previous part of the series and have internalized the best practices described in it. But what do you do about the tons of reports created in the olden days that are now sitting in ~~that huge useless pile~~ your backlog?

What we’ve got on ~~pile~~ file:

Already fixed bugs (since you have such a big backlog, you clearly haven’t been pruning it and it must be full of duplicates).
No longer relevant reports (the functionality no longer exists).
No longer relevant reports (the affected user/client base no longer exists).
No longer relevant reports (bugs that got side-fixed together with other problems).
No longer relevant reports (the platform is no longer supported).
Stealthily fixed bugs (these are even worse than collaterally-fixed ones).
Bugs that keep you awake at night, which you carefully squeeze into every planned release, but there is never enough time for them.
Low-priority bugs that are more expensive to fix than to let be.
Features that need to be done.

What you can do about it:

Link duplicates of already fixed bugs to their resolved counterparts and close them.
When the functionality no longer exists: write a comment (“starting from version … this functionality no longer exists”) and close the bug.
Hard-to-reproduce bugs: if you can’t reproduce it and it’s an old problem (the definition of “old” depends on your project’s dynamics, though) – set the status to “Cannot reproduce,” leave a comment mentioning the version you tested and the environment, then close the bug.
The bug can no longer be reproduced: set the status to “Cannot reproduce,” leave a comment mentioning the version you tested and the environment, and write “possibly fixed together with … such-and-such bug.”
If you located the commit in which the bug was fixed, mark it as “Resolved” and leave a comment specifying the tested version and the environment. And I’ve got news for you: you should start keeping track of your commits.
Update priorities.
Low-priority bugs that are more expensive to fix than to let be: yes, I do think such problems should be reported. You can create a separate label for them in JIRA (or elsewhere). If you don’t do that, testers will bug (pun intended :)) your developers each time they stumble on such a bug. Every time a new person joins your team, they will bother others because they don’t know the backstory. Problems will also arise when it’s been a while and everyone on the testing team has forgotten which verdict they arrived at last time they touched the bug. Moreover, things may change and the bug may suddenly become a priority. Hence, if you report low-priority bugs and keep those reports up to date, it’ll allow you to quickly pull out all communications history when someone requests it.
You’d better decide right away when this can be done and set a version accordingly.

Now let’s see who should be responsible for cleaning up this mess.

Approach 1: Full-on

tackle software bug reports overflow

Just pick someone (one person) for this task. It’s not worth creating a separate position for picking up the garbage created in the testing process.

When this may work: A new tester joins the team and needs to get familiar with the project, and the only test documentation you have is that huge pile of old bug reports.

Sounds like a great idea, but the outcome is absolutely unpredictable – the new employee may not be well-qualified for this; they could bother more experienced testers who may not know the whole story/nuances; the newbie may have trouble reproducing the bug and erroneously close the report without a fix; they may be unable to set adequate priories.

To minimize the negative effect, do a review of all the changes the newcomer makes. Consider it the time you would have invested in training them anyway. Decide who will be responsible for the review and how much time they can spend on it. Have the newbie send a list of the bugs they closed with a “Cannot reproduce” resolution (it’s also recommended to specify the environment) to their supervisor at the end of the day.

Approach 2: Non-intrusive

an illustration of how to tackle backlog grooming

When doing regression/acceptance testing or when testing the current software version, your testing team encounters a problem and checks if it’s been already reported in the backlog. If it has, the tester who found the report adds the version where they had a fresh sighting of the bug and updates the priority.

Approach 3: Mixed (“collective responsibility”)

tackle software bug reports overflow

The newcomer is also in 🙂

Any team member who encounters an already-reported bug comments whether the bug is still reproduced in the current version. If the bug is not reproduced, the tester closes the report.
Pay attention to the bug reports that are being entered into the tracking system. Before reporting a bug, use the search functionality to avoid creating a duplicate. Well-produced and well-maintained bug reports can make up for the discrepancies in your process, especially in a dynamic team.

Quality bug reports can serve as a source of requirements. These are the kind of problems with the process that are cheaper to prevent than to act upon.

Do you have questions or suggestions about this article? If you do, please write them in the comments.

Also read: Part 1: Reporting software bugs like a ninja

Also read: Part 3: What makes for a perfect bug report