GUICop: Approach and Toolset for Specification-based GUI Testing

TL;DR

GUICop introduces a specification-based approach and toolset for GUI testing that verifies GUI behavior against user-defined specifications, effectively handling variations without relying on image processing or fuzzy comparisons.

Contribution

It presents a novel GUI testing framework with a dedicated specification language and toolset that improves accuracy and efficiency over existing fuzzy or image-based methods.

Findings

Successfully tested on four open source GUI applications.

Detected eight defects across the evaluated applications.

Provides a formal, specification-driven approach to GUI testing.

Abstract

Oracles used for testing graphical user interface (GUI) programs are required to take into consideration complicating factors such as variations in screen resolution or color scheme when comparing observed GUI elements to expected GUI elements. Researchers proposed fuzzy comparison rules and computationally expensive image processing techniques to tame the comparison process since otherwise the naive matching comparison would be too constraining and consequently impractical. Alternatively, this paper proposes GUICop, a novel approach with a supporting toolset that takes (1) a GUI program and (2) user-defined GUI specifications characterizing the rendering behavior of the GUI elements, and checks whether the execution traces of the program satisfy the specifications. GUICop comprises the following: 1) a GUI Specification Language; 2) a Driver; 3) Instrumented GUI Libraries; 4) a Solver; and 5) a Code Weaver. The user defines the specifications of the subject GUI program using the GUI Specification Language. The Driver traverses the GUI structure of the program and generates events that drive its execution. The Instrumented GUI Libraries capture the GUI execution trace, i.e., information about the positions and visibility of the GUI elements. And the Solver, enabled by code injected by the Code Weaver, checks whether the traces satisfy the specifications. GUICop was successfully evaluated using four open source GUI applications that included eight defects, namely, Jajuk, Gason, JEdit, and TerpPaint.