Towards Automated Accessibility Report Generation for Mobile Apps
Amanda Swearngin, Jason Wu, Xiaoyi Zhang, Esteban Gomez, Jen, Coughenour, Rachel Stukenborg, Bhavya Garg, Greg Hughes, Adriana Hilliard,, Jeffrey P. Bigham, Jeffrey Nichols

TL;DR
This paper presents an automated system for generating comprehensive accessibility reports for mobile apps by integrating multiple data collection methods, advanced de-duplication, and summarization techniques, improving upon existing single-screen scanners.
Contribution
The authors introduce a novel system that combines app crawling, manual recording, and accessibility scanning with new de-duplication and summarization models for whole app accessibility reporting.
Findings
Screen grouping model with 96.9% accuracy
UI element matching with 97% accuracy
Qualitative evaluation with 18 engineers shows system effectiveness
Abstract
Many apps have basic accessibility issues, like missing labels or low contrast. Automated tools can help app developers catch basic issues, but can be laborious or require writing dedicated tests. We propose a system, motivated by a collaborative process with accessibility stakeholders at a large technology company, to generate whole app accessibility reports by combining varied data collection methods (e.g., app crawling, manual recording) with an existing accessibility scanner. Many such scanners are based on single-screen scanning, and a key problem in whole app accessibility reporting is to effectively de-duplicate and summarize issues collected across an app. To this end, we developed a screen grouping model with 96.9% accuracy (88.8% F1-score) and UI element matching heuristics with 97% accuracy (98.2% F1-score). We combine these technologies in a system to report and summarize…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDigital Accessibility for Disabilities · Tactile and Sensory Interactions · Software Engineering Research
