OwlEyes-Online: A Fully Automated Platform for Detecting and Localizing UI Display Issues

TL;DR

OwlEyes-Online is an automated platform that detects and localizes UI display issues in app screenshots, helping developers identify problems like overlaps or missing images efficiently.

Contribution

The paper introduces OwlEyes-Online, a fully automated online tool for detecting and localizing UI display issues in mobile applications, with automatic app execution and detailed issue reporting.

Findings

High accuracy in detecting UI display issues

Automated localization of problematic UI areas

Generates comprehensive test reports for developers

Abstract

Graphical User Interface (GUI) provides visual bridges between software apps and end users. However, due to the compatibility of software or hardware, UI display issues such as text overlap, blurred screen, image missing always occur during GUI rendering on different devices. Because these UI display issues can be found directly by human eyes, in this paper, we implement an online UI display issue detection tool OwlEyes-Online, which provides a simple and easy-to-use platform for users to realize the automatic detection and localization of UI display issues. The OwlEyes-Online can automatically run the app and get its screenshots and XML files, and then detect the existence of issues by analyzing the screenshots. In addition, OwlEyes-Online can also find the detailed area of the issue in the given screenshots to further remind developers. Finally, OwlEyes-Online will automatically generate test reports with UI display issues detected in app screenshots and send them to users. The OwlEyes-Online was evaluated and proved to be able to accurately detect UI display issues. Tool Link: http://www.owleyes.online:7476 Github Link: https://github.com/franklinbill/owleyes Demo Video Link: https://youtu.be/002nHZBxtCY