Visual Testing of GUIs by Abstraction

TL;DR

This paper introduces an abstract GUI state (AGS) approach for visual testing of GUIs, which focuses on structural relations to detect relevant visual changes, reducing false positives common in pixel-based methods.

Contribution

The paper presents a novel AGS-based method for visual GUI testing that improves change detection accuracy and provides better diagnostic information, supporting fully automated testing.

Findings

Reliable detection of GUI changes in web pages

Reduced false positives compared to pixel-based methods

Supports fully automated visual testing

Abstract

Ensuring the correct visual appearance of graphical user interfaces (GUIs) is important because visual bugs can cause substantial losses for businesses. An application might behave functionally correct in an automated test, but visual bugs can make the GUI effectively unusable for the user. Most of today's approaches for visual testing are pixel-based and tend to have flaws that are characteristic for image differencing. For instance, minor and unimportant visual changes often cause false positives, which confuse the user with unnecessary error reports. Our idea is to introduce an abstract GUI state (AGS), where we define structural relations to identify relevant GUI changes and ignore those that are unimportant from the user's point of view. In addition, we explore several strategies to address the GUI element identification problem in terms of AGS. This allows us to provide rich diagnostic information that help the user to better interpret changes. Based on the principles of golden master testing, we can support a fully-automated approach to visual testing by using the AGS. We have implemented our approach to visually test web pages and our experiments show that we are able to reliably detect GUI changes.