Formula Transformers and Combinatorial Test Generators for Propositional Intuitionistic Theorem Provers

TL;DR

This paper introduces combinatorial test generation algorithms for propositional intuitionistic logic theorem provers, supporting exhaustive and random formula generation, and includes techniques for creating both easier and harder formulas to test prover robustness.

Contribution

It presents novel algorithms for generating formulas in intuitionistic propositional logic, including canonical forms and transformations to test theorem prover correctness.

Findings

Effective formula generators for various intuitionistic logics

Algorithms for reducing formulas to simpler structures

Methods for generating harder formulas to find bugs

Abstract

We develop combinatorial test generation algorithms for progressively more powerful theorem provers, covering formula languages ranging from the implicational fragment of intuitionistic logic to full intuitionistic propositional logic. Our algorithms support exhaustive and random generators for formulas of these logics. To provide known-to-be-provable formulas, via the Curry-Howard formulas-as-types correspondence, we use generators for typable lambda terms and combinator expressions. Besides generators for several classes of formulas, we design algorithms that restrict formula generation to canonical representatives among equiprovable formulas and introduce program transformations that reduce formulas to equivalent formulas of a simpler structure. The same transformations, when applied in reverse, create harder formulas that can catch soundness or incompleteness bugs. To test the effectiveness of the testing framework itself, we describe use cases for deriving lightweight theorem provers for several of these logics and for finding bugs in known theorem provers. Our Prolog implementation available at: https://github.com/ptarau/TypesAndProofs and a subset of formula generators and theorem provers, implemented in Python is available at: https://github.com/ptarau/PythonProvers. Keywords: term and formula generation algorithms, Prolog-based theorem provers, formulas-as-types, type inference and type inhabitation, combinatorial testing, finding bugs in theorem provers.