Identification of Failure Regions for Programs with Numeric Inputs

TL;DR

This paper introduces a new strategy called Search for Boundary (SB) for identifying approximate failure regions in numeric input domains, enhancing testing and debugging processes by effectively locating failure-causing inputs near failure boundaries.

Contribution

The paper proposes a novel IFR strategy, SB, with two specific methods, FSB and DSB, supported by an automated platform, to improve failure region identification in software testing.

Findings

SB methods effectively identify failure regions within limited resources.

Experimental results demonstrate the accuracy of SB in various failure region types.

SB methods outperform existing approaches in boundary detection accuracy.

Abstract

Failure region, where failure-causing inputs reside, has provided many insights to enhance testing effectiveness of many testing methods. Failure region may also provide some important information to support other processes such as software debugging. When a testing method detects a software failure, indicating that a failure-causing input is identified, the next important question is about how to identify the failure region based on this failure-causing input, i.e., Identification of Failure Regions (IFR). In this paper, we introduce a new IFR strategy, namely Search for Boundary (SB), to identify an approximate failure region of a numeric input domain. SB attempts to identify additional failure-causing inputs that are as close to the boundary of the failure region as possible. To support SB, we provide a basic procedure, and then propose two methods, namely Fixed-orientation Search for Boundary (FSB) and Diverse-orientation Search for Boundary (DSB). In addition, we implemented an automated experimentation platform to integrate these methods. In the experiments, we evaluated the proposed SB methods using a series of simulation studies andempirical studies with different types of failure regions. The results show that our methods can effectively identify a failure region, within the limited testing resources.