Death by a thousand bug reports – Part 1: Reporting software bugs like a ninja

Teach everyone on your team to write and manage concise bug reports that save you time and money.

Your software, like any program in the world, contains defects. Defects cause problems in the software’s work. They may be detected by testers, programmers, or people who use your software for work, leisure, or other purposes.

Let’s assume that these people genuinely love your product and can’t wait to report the problem. And that your desire to provide a stellar user experience is no less honest. I think now it’s high time to say that simply detecting the problem is not enough.

Imagine that you want to do a list of chores for the coming week. And because you’re super busy, you are writing down everything that comes to mind on the first surface that meets the eye.

Now your walls, the floor, the ceiling, your cabinets, and even your neighbor’s car parked across the street – everything is covered in scribbles. But will such a list help you? Are you sure you won’t miss anything in this way? Not to mention prioritizing the chores.

Surprise, surprise, this is exactly what some people do with the problems they find in their software. To avoid chaos and to ensure the problem is fixed, you should report it to the right person and provide the necessary details. The “right person” should then prioritize the problems, fix them and hand down the amended program to the awaiting users. The algorithm is as follows:

1. Detecting it

Variant 1 – problem discovered during development
Oops, you’ve got a bug. First and foremost – do not panic, since neither the shareholders nor the users know about it yet.

Bad practice: Keeping it to yourself. Laying low for a few days and/or taking a vacation. Hoping that it will pass. Denying having seen unusual behavior in the software. Trying to hypnotize witnesses, if there were any. 🙂

Good practice: Reporting the bug. What’s characteristic of this case is that it’s cheaper to fix a problem discovered during the development stage than later on in the process.

Variant 2 – problem discovered in production
There’s a bug in production. Discovered by your shareholders/users.

Bad practice: Switching off your phone. Blacklisting users that complain and deleting their tweets. Postponing making a decision. Pointing out that no one is perfect. Feeling sorry for yourself. Saying the program is not supposed to be used in that way. 🙂

Good practice: Suggesting a workaround, staying in touch, reporting the bug. What’s required in this case is to think fast and take logical steps.

2. Reporting it

What matters is the report itself, as well as how you present it to the world.

Bad practice: You’re too lazy. Or have too much to do. Or not enough time to experiment, poor knowledge of the project, etc. As a result the bug is poorly described. Or you just deсide to choose one of the ineffective reporting methods:

Ineffective method 1: Person to person reporting

Pitfalls:

• The information is easily lost/forgotten.
• There is no history of changes or discussions that took place.
• An attempt to forward the bug quickly turns into “Chinese whispers”.

• It’s impossible to hold anyone accountable for missing the bug.
• Programmers feel annoyed and distracted by poorly-timed communications.

Benefits: High speed, especially if you’re sitting in the same office.

Ineffective method 2: Reporting via chat or messenger

Benefits: Minimum bureaucracy is involved.

Pitfalls:

• Chaotic bug report structure.
• Only the author and the recipient have access to the information (the details are in disparate messages and are easily lost).
• Different wording is used to describe “how big the damage is,” which makes it hard to set priorities.

• Difficulty discussing the bug with several people, because then messages are in multiple chats.
• Difficulty referring to the bug in the future, since no ID is assigned.

Good practice: You use a bug tracking system (JIRA, Bugzilla, etc.)

Pitfalls:

• The system may be too complex and expensive for one’s needs.
• There may be too many fields to fill out.
• There’s a risk that your QA will document every tiny problem, which will result in a flood of bug reports (which is the problem this article series solves!).
• Testers can get the description wrong.

Benefits:

• Clear bug report structure.
• Unique report ID – the bug is easy to track.
• It’s easy to work on fixing the bug as a team.
• Version control: you can pinpoint a software version in which the problem occurred/was fixed.
• No info is lost, forgotten or distorted – all discussions happen in one place.
• Ability to generate reports.

Many thanks to Alesia Krush for the cool illustrations. 😉

3. Prioritizing ⭤  reproducing it

Bad Practice: Setting a random priority to the bug - that is too low (potentially making an important part of the app unusable) or too high (making the developers fix a trivial bug while more important issues go unattended). Assigning the issue to a person who doesn’t have enough knowledge, time, etc. to handle it.

Good practice: Taking into account your customers’ needs and severity when choosing in which order bugs should be fixed. As a result, you spent minimum time making decisions because your priorities are clear to the whole team. This is particularly helpful when you are pressed for time, e.g. you have only one hour of time and don’t want to waste 30 minutes deciding which bug(s) to start with.

4. Fixing the bug

Bad practice: Ignoring bug reports assigned to you. Not fixing the bug the first time around (for a lack of competence, domain expertise, unclear expected result) or breaking the program in another place, which leads to new bugs.

Good practice: Developers carefully fix bugs that are assigned to them. Everyone is genuinely engaged in the process and gets each other’s back. You practice code reviews, unit testing, etc.

5. Verifying the fix

Bad practice: Not running any post-fix tests at all. Forgetting to run regression tests. Forgetting to close the issue or being afraid to reopen a fixed bug report.

Good Practice: Giving a precise estimate before testing is started. Verifying the fix in actual environment. Setting an appropriate status after verification.

6. Deploying the fix

Bad practice: Fixing, testing, and forgetting to deploy the fix to production.

Good practice: Deploying the right version (that contains the fix). Creating release notes.

7. Analyzing the root cause of the problem

Think what could have caused the bug to appear. Possible reasons for failure:

• Not enough dev, testing, or domain expertise.
• Wrong priorities (critical tasks are neglected; trivial tasks like button colors are over-discussed).
• Poor process organization, which leads to:
  • Not enough time for creating the architecture.
  • Not enough time for development.
  • Not enough time for software testing.

Pinpointing reasons for failure will help you see the weak links in your development process and to address the issues to improve it moving forward.

This approach will help decrease the number of bug reports naturally by decreasing bugs in your software. But if you’re already nearly sinking in the amount of bug reports produced by your testers, what do you do?

This is exactly what Part 2 of this series talks about. Be sure to read it and let us know what you think about this part in the comments.