Metamorphic Coverage
Jinsheng Ba, Yuancheng Jiang, Manuel Rigger

TL;DR
Metamorphic Coverage (MC) is a new metric that measures the distinct code executed by test input pairs in metamorphic testing, providing a more accurate and efficient way to evaluate testing effectiveness and detect bugs.
Contribution
This work introduces Metamorphic Coverage, a novel metric for assessing metamorphic testing that outperforms traditional coverage and mutation testing in bug detection and efficiency.
Findings
MC overlaps with 78% of bug-fix locations
MC is 4x more sensitive than line coverage
MC is 359x faster than mutation testing
Abstract
Metamorphic testing is a widely used methodology that examines an expected relation between pairs of executions to automatically find bugs, such as correctness bugs. We found that code coverage cannot accurately measure the extent to which code is validated and mutation testing is computationally expensive for evaluating metamorphic testing methods. In this work, we propose Metamorphic Coverage (MC), a coverage metric that examines the distinct code executed by pairs of test inputs within metamorphic testing. Our intuition is that, typically, a bug can be observed if the corresponding code is executed when executing either test input but not the other one, so covering more differential code covered by pairs of test inputs might be more likely to expose bugs. While most metamorphic testing methods have been based on this general intuition, our work defines and systematically evaluates MC…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
