Algebraic Machine Learning: Learning as computing an algebraic decomposition of a task

TL;DR

This paper introduces an algebraic foundation for machine learning, encoding tasks and data as algebraic axioms, enabling models that generalize and perform comparably to neural networks on standard datasets.

Contribution

It presents a novel algebraic approach to learning, replacing traditional statistics and optimization with algebraic axioms and decompositions for model construction.

Findings

Achieves performance comparable to multilayer perceptrons on datasets like MNIST and CIFAR-10.

Provides a method for formal problem solving, such as finding Hamiltonian cycles, without search.

Demonstrates scalability through model additivity and convergence to underlying data rules.

Abstract

Statistics and Optimization are foundational to modern Machine Learning. Here, we propose an alternative foundation based on Abstract Algebra, with mathematics that facilitates the analysis of learning. In this approach, the goal of the task and the data are encoded as axioms of an algebra, and a model is obtained where only these axioms and their logical consequences hold. Although this is not a generalizing model, we show that selecting specific subsets of its breakdown into algebraic atoms obtained via subdirect decomposition gives a model that generalizes. We validate this new learning principle on standard datasets such as MNIST, FashionMNIST, CIFAR-10, and medical images, achieving performance comparable to optimized multilayer perceptrons. Beyond data-driven tasks, the new learning principle extends to formal problems, such as finding Hamiltonian cycles from their specifications and without relying on search. This algebraic foundation offers a fresh perspective on machine intelligence, featuring direct learning from training data without the need for validation dataset, scaling through model additivity, and asymptotic convergence to the underlying rule in the data.