Directional Non-Commutative Monoidal Embeddings for MNIST

TL;DR

This paper empirically validates a novel non-commutative monoidal embedding framework for image classification, demonstrating its ability to learn task-specific spectral features that outperform fixed Fourier-based embeddings on MNIST.

Contribution

It introduces and empirically tests a directional non-commutative monoidal embedding framework that generalizes Fourier transforms by learning task-specific spectral components.

Findings

Learned embeddings outperform fixed DFT embeddings, especially at lower dimensions.

The framework effectively models image data with compact, high-performance representations.

Performance gap increases as embedding dimensionality decreases.

Abstract

We present an empirical validation of the directional non-commutative monoidal embedding framework recently introduced in prior work~\cite{Godavarti2025monoidal}. This framework defines learnable compositional embeddings using distinct non-commutative operators per dimension (axis) that satisfy an interchange law, generalizing classical one-dimensional transforms. Our primary goal is to verify that this framework can effectively model real data by applying it to a controlled, well-understood task: image classification on the MNIST dataset~\cite{lecun1998gradient}. A central hypothesis for why the proposed monoidal embedding works well is that it generalizes the Discrete Fourier Transform (DFT)~\cite{oppenheim1999discrete} by learning task-specific frequency components instead of using fixed basis frequencies. We test this hypothesis by comparing learned monoidal embeddings against fixed DFT-based embeddings on MNIST. The results show that as the embedding dimensionality decreases (e.g., from 32 to 8 to 2), the performance gap between the learned monoidal embeddings and fixed DFT-based embeddings on MNIST grows increasingly large. This comparison is used as an analytic tool to explain why the framework performs well: the learnable embeddings can capture the most discriminative spectral components for the task. Overall, our experiments confirm that directional non-commutative monoidal embeddings are highly effective for representing image data, offering a compact learned representation that retains high task performance. The code used in this work is available at https://github.com/mahesh-godavarti/directional_composition_mnist.