Correctness Attraction: A Study of Stability of Software Behavior Under Runtime Perturbation

TL;DR

This paper introduces a novel protocol to investigate whether software execution can be perturbed without affecting correctness, revealing a phenomenon called 'correctness attraction' through extensive experimental analysis.

Contribution

It presents the first systematic exploration of software stability under runtime perturbations and establishes a new protocol with perfect oracles for correctness verification.

Findings

Many perturbations do not break correctness in studied programs

Introduces the concept of correctness attraction

Provides the first taxonomy of reasons behind correctness attraction

Abstract

Can the execution of a software be perturbed without breaking the correctness of the output? In this paper, we devise a novel protocol to answer this rarely investigated question. In an experimental study, we observe that many perturbations do not break the correctness in ten subject programs. We call this phenomenon ``correctness attraction''. The uniqueness of this protocol is that it considers a systematic exploration of the perturbation space as well as perfect oracles to determine the correctness of the output. To this extent, our findings on the stability of software under execution perturbations have a level of validity that has never been reported before in the scarce related work. A qualitative manual analysis enables us to set up the first taxonomy ever of the reasons behind correctness attraction.