Exploring the Output of Software Testing Tools through a Visual Comparative Analysis
Brandon Lit, Anthony Maocheia-Ricci, Thomas Driscoll

TL;DR
This study conducts a visual comparative analysis of 50 software testing tools to identify common interface elements, visualization patterns, and formatting trends in test outputs across CLI and GUI environments.
Contribution
It provides the first detailed analysis of shared interface elements and visualization patterns in testing tool outputs, informing future tool development.
Findings
Identified common interface elements across testing tools.
Analyzed how color and formatting are used in test result visualizations.
Revealed trends in output presentation for CLI and GUI testing tools.
Abstract
Software testing is a fundamental process of software development, and prior work has shown that visualizations of test results support testers' decision-making. However, Human-Computer Interaction research on software testing has yet to explore and understand the shared interface elements and patterns in visualization of testing outputs. To address this, we conducted a visual comparative analysis of the output of 50 software testing tools and harnesses (44 with CLI output, 6 with GUI output) across four popular programming languages. Our analysis reveals the common interface elements in software testing tools, how these tools display and visualize test results, as well as the specific make-up of the output. Our findings provide insight on how visual testing output is formatted and how colour is used across both CLI and GUI environments, identifying trends that can be applied by…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
