Accessible or Not? An Empirical Investigation of Android App Accessibility

TL;DR

This paper introduces Xbot, an automated tool for fine-grained collection of accessibility issues in Android apps, enabling detailed empirical analysis of issue characteristics, severity, and fixing status across a large dataset.

Contribution

It presents Xbot, an effective automated exploration tool that collects a comprehensive dataset of accessibility issues, facilitating detailed empirical investigation at the issue level.

Findings

Xbot achieves around 80% activity coverage, outperforming existing tools.

86,767 accessibility issues collected from 2,270 apps.

Insights into issue severity, types, patterns, and fixing status.

Abstract

Mobile apps provide new opportunities to people with disabilities to act independently in the world. Motivated by this trend, researchers have conducted empirical studies by using the inaccessibility issue rate of each page (i.e., screen level) to represent the characteristics of mobile app accessibility. However, there still lacks an empirical investigation directly focusing on the issues themselves (i.e., issue level) to unveil more fine-grained findings, due to the lack of an effective issue detection method and a relatively comprehensive dataset of issues. To fill in this literature gap, we first propose an automated app page exploration tool, named Xbot, to facilitate app accessibility testing and automatically collect accessibility issues by leveraging the instrumentation technique and static program analysis. Owing to the relatively high activity coverage (around 80%) achieved by Xbot when exploring apps, Xbot achieves better performance on accessibility issue collection than existing testing tools such as Google Monkey. With Xbot, we are able to collect a relatively comprehensive accessibility issue dataset and finally collect 86,767 issues from 2,270 unique apps including both closed-source and open-source apps, based on which we further carry out an empirical study from the perspective of accessibility issues themselves to investigate novel characteristics of accessibility issues. Specifically, we extensively investigate these issues by checking 1) the overall severity of issues with multiple criteria, 2) the in-depth relation between issue types and app categories, GUI component types, 3) the frequent issue patterns quantitatively, and 4) the fixing status of accessibility issues.