# Is the Stack Distance Between Test Case and Method Correlated With Test Effectiveness?

## TL;DR

This paper investigates whether the minimal stack distance between test cases and methods correlates with test effectiveness, proposing it as a lightweight predictor for mutation testing outcomes based on an empirical study.

## Contribution

It introduces the minimal stack distance as a new measure for estimating test effectiveness and demonstrates its correlation and predictive power through extensive empirical analysis.

## Key findings

- Correlation between stack distance and test effectiveness up to 0.58.
- Classifier using stack distance predicts mutation results with over 92% precision and recall.
- Stack distance can serve as a lightweight alternative or precursor to mutation testing.

## Abstract

Mutation testing is a means to assess the effectiveness of a test suite and its outcome is considered more meaningful than code coverage metrics. However, despite several optimizations, mutation testing requires a significant computational effort and has not been widely adopted in industry. Therefore, we study in this paper whether test effectiveness can be approximated using a more light-weight approach. We hypothesize that a test case is more likely to detect faults in methods that are close to the test case on the call stack than in methods that the test case accesses indirectly through many other methods. Based on this hypothesis, we propose the minimal stack distance between test case and method as a new test measure, which expresses how close any test case comes to a given method, and study its correlation with test effectiveness. We conducted an empirical study with 21 open-source projects, which comprise in total 1.8 million LOC, and show that a correlation exists between stack distance and test effectiveness. The correlation reaches a strength up to 0.58. We further show that a classifier using the minimal stack distance along with additional easily computable measures can predict the mutation testing result of a method with 92.9% precision and 93.4% recall. Hence, such a classifier can be taken into consideration as a light-weight alternative to mutation testing or as a preceding, less costly step to that.