Wavelet Logic Machines: Learning and Reasoning in the Spectral Domain Without Neural Networks

TL;DR

This paper presents a spectral learning framework using wavelet transforms that replaces neural networks, achieving competitive accuracy with fewer parameters and lower memory usage in vision and language tasks.

Contribution

It introduces a fully spectral, neural network-free approach with learnable wavelet basis selection and nonlinear transformations, offering an efficient alternative to traditional neural models.

Findings

Achieves 89.3% accuracy on GLUE SST-2, close to Transformer baseline

Uses 72% fewer parameters and 58% less peak memory

Operates with linear-time wavelet transforms, reducing inference cost

Abstract

We introduce a fully spectral learning framework that eliminates traditional neural layers by operating entirely in the wavelet domain. The model applies learnable nonlinear transformations, including soft-thresholding and gain-phase modulation, directly to wavelet coefficients. It also includes a differentiable wavelet basis selection mechanism, enabling adaptive processing using families such as Haar, Daubechies, and Biorthogonal wavelets. Implemented in PyTorch with full 3D support, the model maintains a spectral pipeline without spatial convolutions or attention. On synthetic 3D denoising and natural language tasks from the GLUE benchmark, including SST-2 sentiment classification, the model achieves 89.3 percent accuracy, close to a 4-layer Transformer baseline (90.1 percent), while using 72 percent fewer parameters and 58 percent less peak memory. Faster early convergence is observed due to spectral sparsity priors. In contrast to the quadratic complexity of self-attention and large matrix multiplications in Transformers, our approach uses linear-time wavelet transforms and pointwise nonlinearities, significantly reducing inference cost. This yields a compact, interpretable, and efficient alternative to neural models. Our results support the viability of principled spectral learning in both vision and language tasks, offering new directions for model design without overparameterized architectures.