TT-Sparse: Learning Sparse Rule Models with Differentiable Truth Tables

TL;DR

TT-Sparse introduces a neural approach using differentiable truth tables and a novel soft TopK operator to learn sparse, interpretable rule models with high accuracy across diverse datasets.

Contribution

It presents a new neural building block with differentiable truth tables and a soft TopK operator for end-to-end learning of sparse, interpretable rule-based models.

Findings

Achieves superior predictive performance on 28 datasets.

Produces models with lower complexity and high interpretability.

Enables exact symbolic rule extraction via Quine-McCluskey minimization.

Abstract

Interpretable machine learning is essential in high-stakes domains where decision-making requires accountability, transparency, and trust. While rule-based models offer global and exact interpretability, learning rule sets that simultaneously achieve high predictive performance and low, human-understandable complexity remains challenging. To address this, we introduce TT-Sparse, a flexible neural building block that leverages differentiable truth tables as nodes to learn sparse, effective connections. A key contribution of our approach is a new soft TopK operator with straight-through estimation for learning discrete, cardinality-constrained feature selection in an end-to-end differentiable manner. Crucially, the forward pass remains sparse, enabling efficient computation and exact symbolic rule extraction. As a result, each node (and the entire model) can be transformed exactly into compact, globally interpretable DNF/CNF Boolean formulas via Quine-McCluskey minimization. Extensive empirical results across 28 datasets spanning binary, multiclass, and regression tasks show that the learned sparse rules exhibit superior predictive performance with lower complexity compared to existing state-of-the-art methods.