Boolformer: Symbolic Regression of Logic Functions with Transformers

TL;DR

Boolformer is a Transformer-based model capable of symbolic regression of Boolean functions, effectively predicting formulas from full or partial data, and demonstrating potential as an interpretable alternative in real-world applications.

Contribution

Introduces Boolformer, a novel Transformer model for symbolic Boolean function regression, capable of handling incomplete data and outperforming genetic algorithms in speed.

Findings

Successfully predicts formulas from full truth tables.

Performs well with noisy or incomplete data.

Achieves significant speedup over genetic algorithms.

Abstract

We introduce Boolformer, a Transformer-based model trained to perform end-to-end symbolic regression of Boolean functions. First, we show that it can predict compact formulas for complex functions not seen during training, given their full truth table. Then, we demonstrate that even with incomplete or noisy observations, Boolformer is still able to find good approximate expressions. We evaluate Boolformer on a broad set of real-world binary classification datasets, demonstrating its potential as an interpretable alternative to classic machine learning methods. Finally, we apply it to the widespread task of modeling the dynamics of gene regulatory networks and show through a benchmark that Boolformer is competitive with state-of-the-art genetic algorithms, with a speedup of several orders of magnitude. Our code and models are available publicly.