On Minor Bugs, or Death by a Thousand Cuts

Defects like other products in IT have a lifecycle. There are variations but in most places I have seen and worked the lifecycle for defects that are fixed, retested and pass retest is largely as below -

Found -> Logged or Reported -> Triaged, Assigned Severity -> Allocated to Developer -> Reported Resolved -> Retested -> (if pass retest) Closed

This is not the only way. Some testing schools, particularly Rapid Software Development recommend talking with the developer to see if a quick fix can be added efficiently without the need to log or report the issue. One with access to source code and sufficient knowledge may have the development knowledge to implement one's own fix. The one problem with this approach is that these incidents would not appear in defect counts in test summary reports.

In a large number of cases the reporting and triage/severity assignment of a defect are done in the same step. That is a problematic approach and assumes that the tester has the background and knowledge to make the same judgement about defect severity as a user or stakeholder. There are numerous times I have misjudged the user impact of some defect and been subsequently corrected.

Typically defects are graded based on an enumerated severity score. One I use is -

Sev 1 : Blocker.

Catastrophic failure of the application or major functionality. No further testing can proceed. Defect must be resolved immediately before delivery.

Sev 2 : Critical.

A failure of a functionality or loss of data which cannot be worked around. Defect must be resolved before delivery.

Sev 3 : Medium

Failure of a functionality for which a workaround exists. Delivery may be done if PM or stakeholders accept the risk.

Sev 4 : Minor, Trivial

Cosmetic defects. Styling issues, spelling mistakes, layout and minor workflow errors etc.

Defects are then prioritised based on available resources, the impact of other work, timelines and fixes applied to Sev 1 and 2 defects first, then 3 and 4. Defects 3 and 4 may be resolved following a production release dependent on the acceptable risk...

..Except that the pressures of timelines, staffing and resourcing and the need to release other features make it very difficult for us to return to work on these lower severity issues. They often get forgotten - that minor useability issue that customers have to live with or that styling defect that makes the application look somewhat less than professional.

This is a dangerous state for the application and the project. While high severity defects mean that the customer cannot do some task or other and thus do typically get prioritised and fixed quickly, the less severe look-feel and useability defects left unresolved are what will annoy your customers and users in the long term, leave a bad impression of the company and at worst will leave your product as shelfware. These are the things that they grumble about in user groups or forums on the web, mention in app and product reviews and tell their friends and colleagues - each a lost avenue that may overwise have lead to a new customer. Once the complaints and dissatisfaction reach a crescendo these initially small issues become major ones and you will have to deal with them in a fast, probably stressful manner.

So how can we manage these? Some ideas I have seen are -

1) Endeavouring to resolve all of them before the first production release. This is likely to be impractical, negates the idea of priorities and severities in the first place and the sudden redevelopment will require more regression testing (automated or not). However it would get the job done.

2) An agile cycle ringfenced to resolve all outstanding non-severe defects. This is a good approach but not perfect. Agile development cycles centre around the release of features and you would lose a cycle of new features in the process. It will thus require buy-in from the stakeholders of the project, who may themselves not be so concerned about minor defects vs key features.

3) A blended approach of fixes drip-fed over time. This may be particularly good where there already exists a continuous integration and deployment process such that the said fixes can be deployed speedily and seamlessly. However this still impacts on resources otherwise likely to be working onthe development of features and there would still need to be a schedule devised such that all the lower severity defects are resolved as soon as possible.

The above was written as a point of discussion and I would love to know your views.