No More DeLuLu: Physics-Inspired Kernel Networks for Geometrically-Grounded Neural Computation

TL;DR

This paper introduces the yat-product kernel and Neural Matter Networks, a geometrically-grounded neural architecture that unifies kernel methods with neural computation, achieving competitive results on image and language tasks.

Contribution

The paper proposes the yat-product kernel and Neural Matter Networks, replacing traditional nonlinearities with a geometrically-grounded kernel operation, unifying kernel learning and neural architectures.

Findings

NMNs match linear baselines on MNIST

Aether-GPT2 outperforms GPT-2 in validation loss

NMNs exhibit bounded prototype evolution and robustness

Abstract

We introduce the yat-product, a kernel operator combining quadratic alignment with inverse-square proximity. We prove it is a Mercer kernel, analytic, Lipschitz on bounded domains, and self-regularizing, admitting a unique RKHS embedding. Neural Matter Networks (NMNs) use yat-product as the sole non-linearity, replacing conventional linear-activation-normalization blocks with a single geometrically-grounded operation. This architectural simplification preserves universal approximation while shifting normalization into the kernel itself via the denominator, rather than relying on separate normalization layers. Empirically, NMN-based classifiers match linear baselines on MNIST while exhibiting bounded prototype evolution and superposition robustness. In language modeling, Aether-GPT2 achieves lower validation loss than GPT-2 with a comparable parameter budget while using yat-based attention and MLP blocks. Our framework unifies kernel learning, gradient stability, and information geometry, establishing NMNs as a principled alternative to conventional neural architectures.