On the Euclidean Distance Degree of Quadratic Two-Neuron Neural Networks

TL;DR

This paper explores the algebraic geometry of shallow quadratic neural networks, introducing the virtual Euclidean Distance degree and analyzing its properties and dependencies on input dimension and metrics.

Contribution

It defines and studies the virtual Euclidean Distance degree for quadratic two-neuron networks, linking it to intersection theory and providing explicit computational methods.

Findings

The virtual Euclidean Distance degree depends polynomially on input dimension.

The Euclidean Distance degree varies with the choice of metric, including the Bombieri-Weyl metric.

Numerical experiments demonstrate the stability and metric dependence of the invariant.

Abstract

We study the Euclidean Distance degree of algebraic neural network models from the perspective of algebraic geometry. Focusing on shallow networks with two neurons, quadratic activation, and scalar output, we identify the associated neurovariety with the second secant variety of a quadratic Veronese embedding. We introduce and analyze the virtual Euclidean Distance degree, a projective invariant defined as the sum of the polar degrees of the variety, which coincides with the usual Euclidean Distance degree for a generic choice of scalar product. Using intersection theory, Chern-Mather classes, and the Nash blow-up provided by Kempf's resolution, we reduce the computation of the virtual Euclidean Distance degree to explicit intersection numbers on a Grassmannian. Applying equivariant localization, we prove that this invariant depends stably polynomially on the input dimension. Numerical experiments based on homotopy continuation illustrate the dependence of the Euclidean Distance degree on the chosen metric and highlight the distinction between the generic and nongeneric cases, such as the Bombieri-Weyl metric.