Fully Automated Functional Fuzzing of Android Apps for Detecting Non-crashing Logic Bugs

TL;DR

This paper presents Genie, a fully automated fuzzing tool that detects non-crashing functional bugs in Android apps by leveraging independent view properties and metamorphic testing, addressing a significant gap in automated testing.

Contribution

The paper introduces independent view fuzzing, a novel automated approach for detecting non-crashing bugs in Android apps without human tests or oracles, enhancing testing coverage.

Findings

Genie effectively detects diverse non-crashing bugs automatically.

The approach outperforms prior methods limited to crash detection.

Genie requires no human-provided tests or oracles.

Abstract

Android apps are GUI-based event-driven software and have become ubiquitous in recent years. Obviously, functional correctness is critical for an app's success. However, in addition to crash bugs, non-crashing functional bugs (in short as "non-crashing bugs" in this work) like inadvertent function failures, silent user data lost and incorrect display information are prevalent, even in popular, well-tested apps. These non-crashing functional bugs are usually caused by program logic errors and manifest themselves on the graphic user interfaces (GUIs). In practice, such bugs pose significant challenges in effectively detecting them because (1) current practices heavily rely on expensive, small-scale manual validation (the lack of automation); and (2) modern fully automated testing has been limited to crash bugs (the lack of test oracles). This paper fills this gap by introducing independent view fuzzing, a novel, fully automated approach for detecting non-crashing functional bugs in Android apps. Inspired by metamorphic testing, our key insight is to leverage the commonly-held independent view property of Android apps to manufacture property-preserving mutant tests from a set of seed tests that validate certain app properties. The mutated tests help exercise the tested apps under additional, adverse conditions. Any property violations indicate likely functional bugs for further manual confirmation. We have realized our approach as an automated, end-to-end functional fuzzing tool, Genie. Given an app, (1) Genie automatically detects non-crashing bugs without requiring human-provided tests and oracles (thus fully automated); and (2) the detected non-crashing bugs are diverse (thus general and not limited to specific functional properties), which set Genie apart from prior work.