Towards a Model of Testers' Cognitive Processes: Software Testing as a Problem Solving Approach

TL;DR

This paper proposes the first cognitive model of software testers' problem solving processes during test case creation, supported by an initial experiment using verbal protocol analysis with students.

Contribution

It introduces a novel cognitive theory of software testing based on problem solving, filling a gap in software engineering psychology.

Findings

Supports a problem solving-based model of test design

Initial experimental results validate the cognitive model

Uses verbal protocol analysis to understand tester decision-making

Abstract

Software testing is a complex, intellectual activity based (at least) on analysis, reasoning, decision making, abstraction and collaboration performed in a highly demanding environment. Naturally, it uses and allocates multiple cognitive resources in software testers. However, while a cognitive psychology perspective is increasingly used in the general software engineering literature, it has yet to find its place in software testing. To the best of our knowledge, no theory of software testers' cognitive processes exists. Here, we take the first step towards such a theory by presenting a cognitive model of software testing based on how problem solving is conceptualized in cognitive psychology. Our approach is to instantiate a general problem solving process for the specific problem of creating test cases. We then propose an experiment for testing our cognitive test design model. The experiment makes use of verbal protocol analysis to understand the mechanisms by which human testers choose, design, implement and evaluate test cases. An initial evaluation was then performed with five software engineering master students as subjects. The results support a problem solving-based model of test design for capturing testers' cognitive processes.