Automated, Cost-effective, and Update-driven App Testing

TL;DR

This paper introduces ATUA, a model-based testing approach that efficiently increases code coverage for app updates using fewer test inputs, reducing effort and improving effectiveness over existing methods.

Contribution

ATUA combines static analysis, dynamic refinement, and multiple testing strategies to target only updated code, significantly enhancing update testing efficiency.

Findings

ATUA achieves higher coverage with fewer inputs.

It reduces testing effort compared to state-of-the-art methods.

Empirical results show improved effectiveness on Android apps.

Abstract

Apps' pervasive role in our society led to the definition of test automation approaches to ensure their dependability. However, state-of-the-art approaches tend to generate large numbers of test inputs and are unlikely to achieve more than 50% method coverage. In this paper, we propose a strategy to achieve significantly higher coverage of the code affected by updates with a much smaller number of test inputs, thus alleviating the test oracle problem. More specifically, we present ATUA, a model-based approach that synthesizes App models with static analysis, integrates a dynamically-refined state abstraction function and combines complementary testing strategies, including (1) coverage of the model structure, (2) coverage of the App code, (3) random exploration, and (4) coverage of dependencies identified through information retrieval. Its model-based strategy enables ATUA to generate a small set of inputs that exercise only the code affected by the updates. In turn, this makes common test oracle solutions more cost-effective as they tend to involve human effort. A large empirical evaluation, conducted with 72 App versions belonging to nine popular Android Apps, has shown that ATUA is more effective and less effort intensive than state-of-the-art approaches when testing App updates.