Towards Human-Like Automated Test Generation: Perspectives from Cognition and Problem Solving

TL;DR

This paper proposes a cognitive science-inspired framework to understand human test creation, aiming to develop automated test generation systems that mimic human problem-solving to better support testers.

Contribution

It introduces a novel framework based on cognition and problem-solving analysis to guide the development of human-like automated test generation tools.

Findings

Framework maps human test design steps and criteria

Enhances understanding of effective human testing strategies

Aims to improve automated test generation to support human testers

Abstract

Automated testing tools typically create test cases that are different from what human testers create. This often makes the tools less effective, the created tests harder to understand, and thus results in tools providing less support to human testers. Here, we propose a framework based on cognitive science and, in particular, an analysis of approaches to problem-solving, for identifying cognitive processes of testers. The framework helps map test design steps and criteria used in human test activities and thus to better understand how effective human testers perform their tasks. Ultimately, our goal is to be able to mimic how humans create test cases and thus to design more human-like automated test generation systems. We posit that such systems can better augment and support testers in a way that is meaningful to them.