Ticket Coverage: Putting Test Coverage into Context
Jakob Rott, Rainer Niedermayr, Elmar Juergens, Dennis Pagano

TL;DR
This paper introduces the ticket coverage metric to assess how well code changes within tickets are tested, providing transparency and revealing test gaps in large industrial systems.
Contribution
It proposes a new metric, ticket coverage, that contextualizes test coverage at the ticket level, addressing a gap in existing testing metrics.
Findings
Ticket coverage reveals untested code within tickets.
Applying ticket coverage improves test transparency.
Test gaps are identified in large industrial codebases.
Abstract
There is no metric that determines how well the implementation of a ticket has been tested. As a consequence, code changed within the context of a ticket might unintentionally remain untested and get into production. This is a major problem, because changed code is more fault-prone than unchanged code. In this paper, we introduce the metric ticket coverage which puts test coverage into the context of tickets. For each ticket, it determines the ratio of changed methods covered by automated or manual tests. We conducted an empirical study on an industrial system consisting of 650k lines of Java code and show that ticket coverage brings transparency into the test state of tickets and reveals relevant test gaps.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Testing and Debugging Techniques · Software Engineering Research · Software Reliability and Analysis Research
