What do all these Buttons do? Statically Mining Android User Interfaces at Scale
Konstantin Kuznetsov (1), Chen Fu (2, 3), Song Gao (2, 3), David, N. Jansen (2), Lijun Zhang (3), Andreas Zeller (1) ((1) CISPA Helmholtz, Center for Information Security, (2) State Key Laboratory of Computer, Science, Institute of Software, Chinese Academy of Sciences

TL;DR
FRONTMATTER is a static analysis tool that automatically extracts detailed UI models and behaviors from Android apps at scale, enabling large-scale empirical research on mobile interfaces.
Contribution
It introduces a novel static analysis approach for comprehensive UI and API extraction from thousands of Android apps, facilitating large-scale UI mining.
Findings
Enabled analysis of tens of thousands of apps
Identified UI and API usage patterns at scale
Facilitated research on accessibility and registration requirements
Abstract
We introduce FRONTMATTER: a tool to automatically mine both user interface models and behavior of Android apps at a large scale with high precision. Given an app, FRONTMATTER statically extracts all declared screens, the user interface elements, their textual and graphical features, as well as Android APIs invoked by interacting with them. Executed on tens of thousands of real-world apps, FRONTMATTER opens the door for comprehensive mining of mobile user interfaces, jumpstarting empirical research at a large scale, addressing questions such as "How many travel apps require registration?", "Which apps do not follow accessibility guidelines?", "Does the user interface correspond to the description?", and many more. FRONTMATTER and the mined dataset are available under an open-source license.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Malware Detection Techniques · Web Data Mining and Analysis · Software Testing and Debugging Techniques
