Maintenance of Automated Test Suites in Industry: An Empirical study on Visual GUI Testing
Emil Al\'egroth, Robert Feldt, Pirjo Kolstr\"om

TL;DR
This empirical study investigates the maintenance costs and influencing factors of automated GUI testing in industry, revealing key cost drivers and providing a model to estimate ROI for test automation.
Contribution
It provides the first detailed empirical analysis of GUI test maintenance costs and factors in industrial practice, along with a cost model for ROI estimation.
Findings
Maintenance costs are influenced by tester experience and test complexity.
Frequent maintenance reduces overall costs compared to infrequent, large updates.
Test automation can lower development costs but requires sufficient initial manual testing effort.
Abstract
Context: Verification and validation (V&V) activities make up 20 to 50 percent of the total development costs of a software system in practice. Test automation is proposed to lower these V&V costs but available research only provides limited empirical data from industrial practice about the maintenance costs of automated tests and what factors affect these costs. In particular, these costs and factors are unknown for automated GUI-based testing. Objective: This paper addresses this lack of knowledge through analysis of the costs and factors associated with the maintenance of automated GUI-based tests in industrial practice. Method: An empirical study at two companies, Siemens and Saab, is reported where interviews about, and empirical work with, Visual GUI Testing is performed to acquire data about the technique's maintenance costs and feasibility. Results: 13 factors are observed…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Engineering Research · Software Engineering Techniques and Practices · Software Reliability and Analysis Research
