PyVeritas: On Verifying Python via LLM-Based Transpilation and Bounded Model Checking for C

TL;DR

PyVeritas introduces a framework that uses LLMs to transpile Python to C, enabling formal verification with existing C model checkers, thus bridging the gap in Python's formal verification tools.

Contribution

The paper presents a novel LLM-based transpilation approach from Python to C combined with bounded model checking, enhancing Python verification capabilities.

Findings

Achieves 80-90% transpilation accuracy with LLMs.

Enables assertion-based verification and fault diagnosis for Python.

Demonstrates effectiveness on small Python programs.

Abstract

Python has become the dominant language for general-purpose programming, yet it lacks robust tools for formal verification. In contrast, programmers working in languages such as C benefit from mature model checkers, for example CBMC, which enable exhaustive symbolic reasoning and fault localisation. The inherent complexity of Python, coupled with the verbosity and low-level nature of existing transpilers (e.g., Cython), have historically limited the applicability of formal verification to Python programs. In this paper, we propose PyVeritas, a novel framework that leverages Large Language Models (LLMs) for high-level transpilation from Python to C, followed by bounded model checking and MaxSAT-based fault localisation in the generated C code. PyVeritas enables verification and bug localisation for Python code using existing model checking tools for C. Our empirical evaluation on two Python benchmarks demonstrates that LLM-based transpilation can achieve a high degree of accuracy, up to 80--90% for some LLMs, enabling effective development environment that supports assertion-based verification and interpretable fault diagnosis for small yet non-trivial Python programs.