Enhancing POI testing approach through the use of additional information

TL;DR

This paper improves the POI testing approach by incorporating common stack traces and advanced call tracing, enabling earlier error detection and new comparison modes for better bug identification.

Contribution

The paper introduces enhancements to POI testing using stack traces and sophisticated call tracing, improving error detection and expanding comparison capabilities.

Findings

Earlier error detection due to enhanced tracing

New comparison modes introduced

Ability to identify unexpected behaviors more easily

Abstract

Recently, a new approach to perform regression testing has been defined: the point of interest (POI) testing. A POI, in this context, is any expression of a program. The approach receives as input a set of relations between POIs from a version of a program and POIs from another version, and also a sequence of input functions, i.e. test cases. Then, a program instrumentation, an input test case generation and different comparison functions are used to obtain the final report which indicates whether the alternative version of the program behaves as expected, e.g. it produces the same values or it uses less CPU/memory. In this paper, we explain how we can improve the POI testing approach through the use of common stack traces and a more sophisticated tracing for calls. These enhancements of the approach allow users to identify errors earlier and easier. Additionally, they enable new comparison modes and new categories of reported unexpected behaviours.