Polynomial Surrogate Training for Differentiable Ternary Logic Gate Networks

TL;DR

This paper introduces Polynomial Surrogate Training (PST), a novel method for training differentiable ternary logic gate networks that significantly reduces parameters and enables effective learning of ternary circuits with uncertainty handling.

Contribution

PST represents ternary neurons as degree-2 polynomials, reducing parameters and bounding the gap to discrete logic, facilitating scalable and effective ternary logic network training.

Findings

Ternary networks train 2-3 times faster than binary DLGNs.

Networks discover diverse true ternary gates.

UNKNOWN state enables effective uncertainty-based selective prediction.

Abstract

Differentiable logic gate networks (DLGNs) learn compact, interpretable Boolean circuits via gradient-based training, but all existing variants are restricted to the 16 two-input binary gates. Extending DLGNs to Ternary Kleene $K_3$ logic and training DTLGNs where the UNKNOWN state enables principled abstention under uncertainty is desirable. However, the support set of potential gates per neuron explodes to $19{,}683$, making the established softmax-over-gates training approach intractable. We introduce Polynomial Surrogate Training (PST), which represents each ternary neuron as a degree-$(2,2)$ polynomial with 9 learnable coefficients (a $2{,}187\times$ parameter reduction) and prove that the gap between the trained network and its discretized logic circuit is bounded by a data-independent commitment loss that vanishes at convergence. Scaling experiments from 48K to 512K neurons on CIFAR-10 demonstrate that this hardening gap contracts with overparameterization. Ternary networks train $2$-$3\times$ faster than binary DLGNs and discover true ternary gates that are functionally diverse. On synthetic and tabular tasks we find that the UNKNOWN output acts as a Bayes-optimal uncertainty proxy, enabling selective prediction in which ternary circuits surpass binary accuracy once low-confidence predictions are filtered. More broadly, PST establishes a general polynomial-surrogate methodology whose parameterization cost grows only quadratically with logic valence, opening the door to many-valued differentiable logic.