Programmable Property-Based Testing
Alperen Keles, Justine Frank, Ceren Mert, Harrison Goldstein, Leonidas Lampropoulos

TL;DR
This paper introduces a new programmable property-based testing language that decouples property definitions from execution, enabling rapid prototyping of domain-specific testing frameworks with increased flexibility.
Contribution
It proposes a deeper language for properties using deferred binding abstract syntax, implemented in Rocq and Racket, allowing customizable and domain-specific testing approaches.
Findings
Decoupling properties from runners enhances flexibility.
Rapid prototyping of domain-specific testing frameworks is feasible.
Programmable properties improve testing customization.
Abstract
Property-based testing (PBT) is a popular technique for establishing confidence in software, where users write properties -- i.e., executable specifications -- that can be checked many times in a loop by a testing framework. In modern PBT frameworks, properties are usually written in shallowly embedded domain-specific languages, and their definition is tightly coupled to the way they are tested. Such frameworks often provide convenient configuration options to customize aspects of the testing process, but users are limited to precisely what library authors had the prescience to allow for when developing the framework; if they want more flexibility, they may need to write a new framework from scratch. We propose a new, deeper language for properties based on a mixed embedding that we call deferred binding abstract syntax, which reifies properties as a data structure and decouples them…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Testing and Debugging Techniques · Logic, programming, and type systems · Software Engineering Research
