Symbolic Branch Networks: Tree-Inherited Neural Models for Interpretable Multiclass Classification

TL;DR

Symbolic Branch Networks (SBNs) are neural models derived from decision trees that combine interpretability with competitive predictive performance through a semi-symbolic learning approach.

Contribution

This work introduces SBN, a semi-symbolic neural model that preserves tree-based interpretability while improving accuracy via gradient-based learning of feature-to-branch mappings.

Findings

SBN matches or exceeds XGBoost on 28 datasets.

SBN* achieves competitive results with no trainable symbolic parameters.

SBN maintains interpretability with strong predictive performance.

Abstract

Symbolic Branch Networks (SBNs) are neural models whose architecture is inherited directly from an ensemble of decision trees. Each root-to-parent-of-leaf decision path is mapped to a hidden neuron, and the matrices $W_{1}$ (feature-to-branch) and $W_{2}$ (branch-to-class) encode the symbolic structure of the ensemble. Because these matrices originate from the trees, SBNs preserve transparent feature relevance and branch-level semantics while enabling gradient-based learning. The primary contribution of this work is SBN, a semi-symbolic variant that preserves branch semantics by keeping $W_{2}$ fixed, while allowing $W_{1}$ to be refined through learning. This controlled relaxation improves predictive accuracy without altering the underlying symbolic structure. Across 28 multiclass tabular datasets from the OpenML CC-18 benchmark, SBN consistently matches or surpasses XGBoost while retaining human-interpretable branch attributions. We also analyze SBN*, a fully symbolic variant in which both $W_{1}$ and $W_{2}$ are frozen and only calibration layers are trained. Despite having no trainable symbolic parameters, SBN* achieves competitive performance on many benchmarks, highlighting the strength of tree-derived symbolic routing as an inductive bias. Overall, these results show that symbolic structure and neural optimization can be combined to achieve strong performance while maintaining stable and interpretable internal representations.