On Extracting Unit Tests from Interactive Programming Sessions

TL;DR

This paper introduces a method to automatically extract unit tests from exploratory programming sessions by recording developer interactions and applying machine learning clustering, aiming to improve automated test generation.

Contribution

It proposes a novel approach to leverage exploratory testing bursts for automatic test extraction using environment wiretapping and machine learning clustering techniques.

Findings

Prototype implementations for static and dynamic languages

Real-time extraction of scripted tests from recordings

Integration with programming by example research direction

Abstract

Software engineering methodologies propose that developers should capture their efforts in ensuring that programs run correctly in repeatable and automated artifacts, such as unit tests. However, when looking at developer activities on a spectrum from exploratory testing to scripted testing we find that many engineering activities include bursts of exploratory testing. In this paper we propose to leverage these exploratory testing bursts by automatically extracting scripted tests from a recording of these sessions. In order to do so, we wiretap the development environment so we can record all program input, all user-issued functions calls, and all program output of an exploratory testing session. We propose to then use machine learning (i.e. clustering) to extract scripted test cases from these recordings in real-time. We outline two early-stage prototypes, one for a static and one for a dynamic language. And we outline how this idea fits into the bigger research direction of programming by example.