Galapagos: Automated N-Version Programming with LLMs

TL;DR

This paper introduces Galápagos, a tool that automates the generation of diverse, functionally equivalent program variants using large language models, enhancing fault tolerance and bug protection in software systems.

Contribution

Galápagos demonstrates automated creation and validation of diverse program variants with LLMs, enabling practical N-Version programming for fault tolerance.

Findings

Galápagos can produce functionally equivalent variants in different languages.

Variants are statically different and behave divergently at runtime.

The approach protects against real compiler miscompilation bugs.

Abstract

N-Version Programming is a well-known methodology for developing fault-tolerant systems. It achieves fault detection and correction at runtime by adding diverse redundancy into programs, minimizing fault mode overlap between redundant program variants. In this work, we propose the automated generation of program variants using large language models. We design, develop and evaluate Gal\'apagos: a tool for generating program variants using LLMs, validating their correctness and equivalence, and using them to assemble N-Version binaries. We evaluate Gal\'apagos by creating N-Version components of real-world C code. Our original results show that Gal\'apagos can produce program variants that are proven to be functionally equivalent, even when the variants are written in a different programming language. Our systematic diversity measurement indicates that functionally equivalent variants produced by Gal\'apagos, are statically different after compilation, and present diverging internal behavior at runtime. We demonstrate that the variants produced by Gal\'apagos can protect C code against real miscompilation bugs which affect the Clang compiler. Overall, our paper shows that producing N-Version software can be drastically automated by advanced usage of practical formal verification and generative language models.