Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometry

TL;DR

This paper advocates for studying neural network function spaces using algebraic geometry, proposing a new research direction called neuroalgebraic geometry that links algebraic invariants to machine learning properties.

Contribution

It introduces the concept of neuroalgebraic geometry, connecting algebraic invariants of polynomial neural networks to key machine learning aspects, and reviews related literature.

Findings

Proposes a dictionary linking algebraic invariants to learning properties.

Highlights potential of algebraic geometry to analyze neural network complexity.

Lays groundwork for a new interdisciplinary research field.

Abstract

In this position paper, we promote the study of function spaces parameterized by machine learning models through the lens of algebraic geometry. To this end, we focus on algebraic models, such as neural networks with polynomial activations, whose associated function spaces are semi-algebraic varieties. We outline a dictionary between algebro-geometric invariants of these varieties, such as dimension, degree, and singularities, and fundamental aspects of machine learning, such as sample complexity, expressivity, training dynamics, and implicit bias. Along the way, we review the literature and discuss ideas beyond the algebraic domain. This work lays the foundations of a research direction bridging algebraic geometry and deep learning, that we refer to as neuroalgebraic geometry.