On the Stable Euclidean Distance Degree of Algebraic Layers

TL;DR

This paper investigates the projective geometry of algebraic neural layers, demonstrating that the generic Euclidean Distance degree is a stable polynomial depending only on the activation polynomial degree, using intersection theory and Schubert calculus.

Contribution

It proves the stable polynomial nature of the Euclidean Distance degree for algebraic neural layers, depending solely on activation polynomial degree, via intersection theory and localization techniques.

Findings

gED is a stable polynomial in input/output dimensions

Stable polynomial depends only on activation polynomial degree

Method uses intersection theory and Schubert calculus

Abstract

We study the projective geometry of algebraic neural layers, namely families of maps induced by a polynomial activation function, with particular emphasis on the generic Euclidean Distance degree ($\mathrm{gED}$). This invariant is projective in nature and measures the number of optimal approximations of a general point in the ambient space with respect to a general metric. For a fixed architecture (i.e. fixed width and activation polynomial), we prove that the $\mathrm{gED}$ is stably polynomial in the dimensions of the input and output spaces. Moreover, we show that this stable polynomial depends only on the degree of the activation function. Our approach relies on standard intersection theory on the Nash blow-up, which allows us to express the $\gED$ as an intersection number over products of Grassmannians. Stable polynomiality is deduced via equivariant localization, while the reduction to the monomial case follows from an explicit Schubert calculus computation on Grassmannians.