On Defect Taxonomies

Presents considerations for designing defect taxonomies and our draft taxonomy.

---

Definition "Defect Taxonomy"

A defect is a structural property of a software document of any kind (e.g. requirements description, test plan, source code, configuration file), namely a deviation from the nearest (i.e. most similar) correct document that makes the document incorrect or locally incorrect.

A taxonomy is a system of hierarchical categories designed to be a useful aid for reproducibly classifying things.

Purpose of a defect taxonomy

The following considerations assume that defects are recorded when they are found throughout the software process, including their classification according to the defect taxonomy. Then a defect taxonomy can serve one or several of the following purposes:

• Understanding process characteristics: Recording defects and classifying them can help understand the process with respect to all those activities that produce defects. In particular, they help in identifying process weaknesses (high-defect steps).
• Understanding product characteristics:
  • Identifying points needing rework: Clusters of defects point to documents that need rework.
  • Suggesting rework approaches: The kinds of defects may point out the best approach for doing the rework (e.g. direct repair, review, redesign, retest, etc.)
• Providing project guidance: Defect characteristics of artifacts may help with risk assessment for process decisions such as task priorities, go/no-go decisions, reimplementation decisions etc.
• ?

Aspects that can be used in forming the taxonomy

Any two taxonomy entries will differ with respect to one or several of the following defect aspects:

• Kind of underlying human error: e.g. oversight, misdeduction, misbelief/ignorance (by the document authors)
• The nature of the most likely repair: addition, deletion, modification (of some part of the document)
• The number of individuals involved in producing the defect: One, more than one.
• The kind of (sub)clause containing the defect: boolean formula, object name, operation name, list, quantifier, quantity, string

Design considerations

The cross-product of all of the above aspects would not lead to a useful taxonomy. The reasons are as follows:

• Very many of the resulting classes would occur essentially never (and would hence be difficult to understand and hence confuse users.

Rather, we use the following approach and priorities for constructing the taxonomies:

• A smaller taxonomy (fewer classes) is preferable over a larger one, because higher rates of misclassification in larger taxonomies will quickly outweigh the usefulness of additional classes.
• Specifically, the use of an aspect should be avoided if the classification according to that aspect will often be difficult
• ?

((to be completed))

Defect taxonomy (draft)

((actual taxonomy is still to be begun))

References

Lots of defect taxonomies and ways how to build and utilize them have been suggested. Here's a short list of some interesting work:

• IEEE Std 1044-1993: Standard Classification for Software Anomalies
• Ram Chillarege, Inderpal S. Bhandari, Jarir K. Chaar, Michael J. Halliday, Diane S. Moebus, Bonnie K. Ray, Man-Yuen Wong. Orthogonal Defect Classification - A Concept for In-Process Measurements. IEEE Transactions on Software Engineering, Vol 18, No. 11, Nov 1992.
• Boris Beizer: Software Testing Techniques. Van Nostrand Reinhold Co., 1990. Includes exhaustive defect taxonomy which can be found online in a modified version.
• Victor R. Basili, Barry T. Perricone: Software errors and complexity: an empirical investigation, Communications of the ACM, Volume 27, Issue 1 (January 1984), Pages: 42 - 52.
• Michael Fredericks, Victor Basili. Using Defect Tracking and Analysis to Improve Software Quality. DACS State-of-the-Art Report SP0700-98-4000, Data & Analysis Center for Software, 14 November 1998
• Brian Marick. A survey of software fault surveys. Technical Report UIUCDCS-R-90-1651, University of Illinois at Urbana Champaign, December 1990.
• Bernd Freimut. Developing and Using Defect Classification Schemes. Report Fraunhofer IESE-072.01/E, Sep. 1, 2001.

(Comments)

If you have comments or additions but do not want to include them in the main body above, put add them here and sign as indicated below the text window.

SWTIDSR