ChimpCheck: Property-Based Randomized Test Generation for Interactive Apps

TL;DR

ChimpCheck is a testing tool that combines scripted and randomized UI testing for Android apps, enabling efficient, relevant, property-based test generation to uncover defects in complex interactive applications.

Contribution

It introduces a high-level DSL and combinator library for property-based randomized testing, bridging manual scripting and automated random testing for Android apps.

Findings

Effective in exposing defects in open source Android apps

Enables compact expression of testing patterns

Bridges manual and automated testing approaches

Abstract

We consider the problem of generating relevant execution traces to test rich interactive applications. Rich interactive applications, such as apps on mobile platforms, are complex stateful and often distributed systems where sufficiently exercising the app with user-interaction (UI) event sequences to expose defects is both hard and time-consuming. In particular, there is a fundamental tension between brute-force random UI exercising tools, which are fully-automated but offer low relevance, and UI test scripts, which are manual but offer high relevance. In this paper, we consider a middle way---enabling a seamless fusion of scripted and randomized UI testing. This fusion is prototyped in a testing tool called ChimpCheck for programming, generating, and executing property-based randomized test cases for Android apps. Our approach realizes this fusion by offering a high-level, embedded domain-specific language for defining custom generators of simulated user-interaction event sequences. What follows is a combinator library built on industrial strength frameworks for property-based testing (ScalaCheck) and Android testing (Android JUnit and Espresso) to implement property-based randomized testing for Android development. Driven by real, reported issues in open source Android apps, we show, through case studies, how ChimpCheck enables expressing effective testing patterns in a compact manner.