Error prevention in software development

In recent years, progress in quality assurance for software products has mainly been made in the area of testing (i.e. executing the software with the aim to cause failures which in turn are signs for defects) and static code analysis. Inspection techniques seek defects in software artefact or any other intermediate document directly. As it is, quality assurance departments in software organizations recognize their job mainly as "hunting for bugs". Of course, after finding such problems they still need to be resolved. It's a strange phenomenon that software engineering seems to accept software defects as to be removed after being created by professionals.

In contrast, our research on error prevention focuses on prediction defects from being inserted in the first place. We assume that

In the first step we focus on the study of coding process, but generally design activities share similar properties. The research is currently guided by three ways of investigation.

Some terminology

In short, errors cause defects (immediately) and defects may cause failures (usually not immediately). To stress the similarity, errors can also be termed as "human failures". Consequently, we add the equivalent to the "human defect", the

Note that the research's subject is "*Error* prevention...". We focus on errors, not on defects. And we focus on suppoting the programmer to learn about her own errors to avoid them later.

In the past, lots of quality aware process improvements have been suggested including Root Cause Analysis, Orthogonal Defect Classification and similar. These focus on analysing defects which have been found during quality assurance with the result of suggestions for process improvements.

Additionally, we benefit from improvements of engineer's tools and techniques, for example

A special case is the Personal Software Process, part of which is concerned with defect analysis which indeed fulfils the four assumptions mentioned above but is proved as being too challenging to implement in software development departments.

Other like the Cleanroom Software Development are based on formal methods to prevent defects from the very beginning of development by reducing errors in the process of translating requirements to computer-executable code.

Capturing the micro-process of software development (activity analysis)

First step is to analyse which activities (in German: Tätigkeiten) are performed during programming (mostly coding; we will analyse design aspects later) to get a feeling for typical patterns of work and typical situations where code was written, which later was classified as defective. In short, with doing micro-process analysis we'll define the vocabulary to describe problematic coding situations. The long-term objective is to automatically detect those situations and warn the programmer just-in-time.

In the first step, we will concentrate on the task of programming. With ongoing investment in formalizing the design process and in using tools to support the construction of well-defined design documents while keeping them in sync with the code, the ideas presented here can be extended to design activities of software development as well.

The vocabulary for describing the micro-process of coding will be based on events occurring on some code part, like

These events will be grouped to typical episodes like

The aim is to log events and episodes and find statistically relevant relations to introduced defects, i.e. identifying typical patterns of defect insertion. For example, typical “anti-patterns” and indicators are

By carefully examining the cause it will be possible to predict errors and therefore prevent resulting defects. At least it will be possible for the developer to reinvestigate how she introduced the defect, i.e. performing some root cause analysis, which may result in some micro-process improvement.

For a more detailed discussion on micro-processes in software development, please see our special pages at MicroprocessHome.

Related work on analysing programmer's activity include:

Psychology of programming and human error

Human errors mark the subset of human actions which are faulty or wrong. Faulty actions simply have been weakly performed although the initial intention was correct. For example, in order to enter a room, one may try to push the door instead of pulling it. Wrong actions on the other hand have been performed well, but were unsuited for the situation they occurred in. For example, opening the door of a car while driving with high speed is a bad idea. Programmers usually perform lots of faulty or wrong activities.

Psychology provides some in-deep studies of human error with the aim to explain why they occur. Although explanation is not our focus, it may give some hints for error prediction. Additionally, errors have been classified, although it's been mainly done for actions (in German: Handlungen) performed by the human body, but human errors in reasoning have also been classified, especially in planning tasks. All this may provide us with good ground to re-phrase these psychological motivated descriptions for the special task of programming. For example, the error provoking effect on interruptions have been studied considerably. As another example, it has been observed that man only remembers 7 +- 2 items in short term memory which may limit the programming activity a lot. So far, only little research has systematically investigated human error and misbelief as a cause for defects in software development.

Amoung others, following references provide deeper understanding.

There's been a lot of psychological studies on the micro-process of coding, code comprehension, design, and others. Please see MicroprocessHome for details.

Classification of failures, defects, errors, and misbeliefs

Classification of defects has been provided by many of the efforts on preventing defects but no agreement on taxonomies exists. A classification will help us focus on important types of defects and errors and gives insight into mechanisms why defects occurred (i.e. which error was performed). The weakness of many classifications is that they do not clearly distinguish between defects and errors. Some even mix up failures and defects. We will provide an alternative DefectTaxonomy which suits our needs, i.e.

The kinds of failure, i.e. bad computation, endless loops, or accessing protected memory are seldom of interest. A lot of effort has been made in classifying defects, although many did not separate defect and human error very well. As mentioned above, defects can be described by categorizing the code change which removes it, covering simple "off-by-one" loop condition defects as well as redesign of complete parts to resolve concurrency problems, for example.

Lots of related work can be presented here, like

Contact

In case of any question or suggestion please get into contact with SebastianJekutsch.

SWTIDSR