Towards Property-Based Tests in Natural Language

TL;DR

This paper introduces a novel method for generating property-based tests directly from natural language descriptions using combinatory categorial grammars, offering an alternative to machine learning or templated approaches.

Contribution

It applies linguistics, specifically CCGs, to translate natural language into executable tests, demonstrating feasibility with textbook examples.

Findings

Prototype successfully generates tests from English descriptions.

Approach offers a linguistics-based alternative to ML and templated methods.

Potential to improve test generation from natural language.

Abstract

We consider a new approach to generate tests from natural language. Rather than relying on machine learning or templated extraction from structured comments, we propose to apply classic ideas from linguistics to translate natural-language sentences into executable tests. This paper explores the application of combinatory categorial grammars (CCGs) to generating property-based tests. Our prototype is able to generate tests from English descriptions for each example in a textbook chapter on property-based testing.