AccessFixer: Enhancing GUI Accessibility for Low Vision Users With R-GCN Model

TL;DR

AccessFixer is a novel R-GCN-based method that automatically repairs GUI accessibility issues like small size, narrow intervals, and low contrast, significantly improving usability for low vision users and gaining developer approval.

Contribution

This work introduces AccessFixer, the first approach to automatically fix multiple GUI accessibility issues simultaneously using R-GCN, with demonstrated effectiveness on real-world and open-source apps.

Findings

Fixes 81.2% of accessibility issues in real-world apps.

8 PRs merged or under fixing on GitHub.

Positive user feedback from low vision users.

Abstract

The Graphical User Interface (GUI) plays a critical role in the interaction between users and mobile applications (apps), aiming at facilitating the operation process. However, due to the variety of functions and non-standardized design, GUIs might have many accessibility issues, like the size of components being too small or their intervals being narrow. These issues would hinder the operation of low vision users, preventing them from obtaining information accurately and conveniently. Although several technologies and methods have been proposed to address these issues, they are typically confined to issue identification, leaving the resolution in the hands of developers. Moreover, it can be challenging to ensure that the color, size, and interval of the fixed GUIs are appropriately compared to the original ones. In this work, we propose a novel approach named AccessFixer, which utilizes the Relational-Graph Convolutional Neural Network (R-GCN) to simultaneously fix three kinds of accessibility issues, including small sizes, narrow intervals, and low color contrast in GUIs. With AccessFixer, the fixed GUIs would have a consistent color palette, uniform intervals, and adequate size changes achieved through coordinated adjustments to the attributes of related components. Our experiments demonstrate the effectiveness and usefulness of AccessFixer in fixing GUI accessibility issues. After fixing 30 real-world apps, our approach solves an average of 81.2% of their accessibility issues. Also, we apply AccessFixer to 10 open-source apps by submitting the fixed results with pull requests (PRs) on GitHub. The results demonstrate that developers approve of our submitted fixed GUIs, with 8 PRs being merged or under fixing. A user study examines that low vision users host a positive attitude toward the GUIs fixed by our method.