Testing Criteria for Mobile Apps Based on Callback Sequences

TL;DR

This paper proposes callback sequence-based testing criteria for mobile apps, focusing on app behaviors beyond GUI actions, to improve bug detection and testing efficiency.

Contribution

It introduces a new family of testing criteria based on callback sequences and a measurement model using CCFA, addressing overlooked app behaviors.

Findings

Testing with callback-based criteria finds more bugs.

Guided testing triggers bugs faster.

Outperforms state-of-the-art tools in bug detection.

Abstract

App quality has been shown to be the most important indicator of app adoption. To assure quality, developers mainly use testing to find bugs in app and apply structural and GUI test coverage criteria. However, mobile apps have more behaviors than the GUI actions, e.g. an app also handles events from sensors and executes long-running background tasks through Android API calls to Services and AsyncTasks. Our studies found that there are important app behaviors via callback interactions that should be covered in testing, as data sharing between callbacks is common and is the cause of many existing bugs. We design a family of test criteria based on callback sequences and use the Callback Control Flow Automata (CCFA) to measure the coverage for testing. Our experiments show that guiding by our criteria, testing can find more bugs and trigger bugs faster than the state-of-the-art tools.