Ternary Gamma Semirings: From Neural Implementation to Categorical Foundations

TL;DR

This paper links neural network learning with algebraic structures, showing that with logical constraints, networks learn structured feature spaces modeled as finite commutative ternary Gamma semirings, explaining their generalization capabilities.

Contribution

It introduces the Ternary Gamma Semiring as a mathematical framework for neural representations, connecting algebraic structures with neural generalization and compositional learning.

Findings

Neural networks fail on compositional generalization without constraints

Logical constraints lead to learning algebraic structures in features

The learned structure is a finite commutative ternary Gamma semiring

Abstract

This paper establishes a theoretical framework connecting neural network learning with abstract algebraic structures. We first present a minimal counterexample demonstrating that standard neural networks completely fail on compositional generalization tasks (0% accuracy). By introducing a logical constraint -- the Ternary Gamma Semiring -- the same architecture learns a perfectly structured feature space, achieving 100% accuracy on novel combinations. We prove that this learned feature space constitutes a finite commutative ternary $\Gamma$-semiring, whose ternary operation implements the majority vote rule. Comparing with the recently established classification of Gokavarapu et al., we show that this structure corresponds precisely to the Boolean-type ternary $\Gamma$-semiring with $|T|=4$, $|\Gamma|=1$}, which is unique up to isomorphism in their enumeration. Our findings reveal three profound conclusions: (i) the success of neural networks can be understood as an approximation of mathematically ``natural'' structures; (ii) learned representations generalize because they internalize algebraic axioms (symmetry, idempotence, majority property); (iii) logical constraints guide networks to converge to these canonical forms. This work provides a rigorous mathematical framework for understanding neural network generalization and inaugurates the new interdisciplinary direction of Computational $\Gamma$-Algebra.