Are your apps accessible? A GCN-based accessibility checker for low vision users

TL;DR

This paper introduces ALVIN, a GCN-based tool that models GUIs as graphs to accurately detect accessibility issues for low vision users, outperforming existing rule-based methods.

Contribution

The paper presents a novel GCN-based approach for GUI accessibility checking that effectively models component relations and reduces redundancy, improving detection accuracy.

Findings

ALVIN achieves 83.5% precision and 78.9% recall in detecting accessibility issues.

ALVIN outperforms baseline methods and other models in experiments.

User studies confirm ALVIN's usefulness in real app development.

Abstract

Context: Accessibility issues (e.g., small size and narrow interval) in mobile applications (apps) lead to obstacles for billions of low vision users in interacting with Graphical User Interfaces (GUIs). Although GUI accessibility scanning tools exist, most of them perform rule-based check relying on complex GUI hierarchies. This might make them detect invisible redundant information, cannot handle small deviations, omit similar components, and is hard to extend. Objective: In this paper, we propose a novel approach, named ALVIN (Accessibility Checker for Low Vision), which represents the GUI as a graph and adopts the Graph Convolutional Neural Networks (GCN) to label inaccessible components. Method: ALVIN removes invisible views to prevent detecting redundancy and uses annotations from low vision users to handle small deviations. Also, the GCN model could consider the relations between GUI components, connecting similar components and reducing the possibility of omission. ALVIN only requires users to annotate the relevant dataset when detecting new kinds of issues. Results: Our experiments on 48 apps demonstrate the effectiveness of ALVIN, with precision of 83.5%, recall of 78.9%, and F1-score of 81.2%, outperforming baseline methods. In RQ2, the usefulness is verified through 20 issues submitted to open-source apps. The RQ3 also illustrates the GCN model is better than other models. Conclusion: To summarize, our proposed approach can effectively detect accessibility issues in GUIs for low vision users, thereby guiding developers in fixing them efficiently.