Testing Updated Apps by Adapting Learned Models

TL;DR

CALM is an automated testing approach that efficiently verifies app updates by adapting learned models, reducing testing resources while increasing coverage of updated code segments.

Contribution

The paper introduces CALM, a novel method for testing app updates by adapting models from previous tests to focus on changed features, improving efficiency and coverage.

Findings

CALM exercises more updated methods than six state-of-the-art approaches.

CALM outperforms competitors especially when few methods are updated.

CALM reduces the number of screens needed for visual inspection.

Abstract

Although App updates are frequent and software engineers would like to verify updated features only, automated testing techniques verify entire Apps and are thus wasting resources. We present Continuous Adaptation of Learned Models (CALM), an automated App testing approach that efficiently test App updates by adapting App models learned when automatically testing previous App versions. CALM focuses on functional testing. Since functional correctness can be mainly verified through the visual inspection of App screens, CALM minimizes the number of App screens to be visualized by software testers while maximizing the percentage of updated methods and instructions exercised. Our empirical evaluation shows that CALM exercises a significantly higher proportion of updated methods and instructions than six state-of-the-art approaches, for the same maximum number of App screens to be visually inspected. Further, in common update scenarios, where only a small fraction of methods are updated, CALM is even quicker to outperform all competing approaches in a more significant way.