An Empirical Study on Failed Error Propagation in Java Programs with Real Faults
Gunel Jahangirova, David Clark, Mark Harman, Paolo Tonella

TL;DR
This study empirically investigates how often error propagation fails in Java programs with real faults, revealing that failure rates are low at unit level but higher at system level, impacting testing strategies.
Contribution
It provides the first comprehensive empirical analysis of failed error propagation in real Java faults across multiple projects, informing testing practices.
Findings
Failed error propagation is negligible at unit testing level.
Failure rates increase significantly at system testing level.
Method postconditions should focus on externally observable state.
Abstract
During testing, developers can place oracles externally or internally with respect to a method. Given a faulty execution state, i.e., one that differs from the expected one, an oracle might be unable to expose the fault if it is placed at a program point with no access to the incorrect program state or where the program state is no longer corrupted. In such a case, the oracle is subject to failed error propagation. We conducted an empirical study to measure failed error propagation on Defects4J, the reference benchmark for Java programs with real faults, considering all 6 projects available (386 real bugs and 459 fixed methods). Our results indicate that the prevalence of failed error propagation is negligible when testing is performed at the unit level. However, when system-level inputs are provided, the prevalence of failed error propagation increases substantially. This indicates…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Testing and Debugging Techniques · Software Reliability and Analysis Research · Software Engineering Research
