Critical Points of Degenerate Metrics on Algebraic Varieties: A Tale of Overparametrization
Giovanni Luca Marchetti, Erin Connelly, Paul Breiding, Kathl\'en Kohn

TL;DR
This paper explores the critical points of degenerate quadratic optimization problems on algebraic varieties, linking algebraic geometry with machine learning, especially in overparametrized models, and extends existing mathematical tools to this setting.
Contribution
It introduces a method to relate degenerate optimization problems to nondegenerate ones via projection, and analyzes the role of ramification loci in highly-degenerate regimes.
Findings
Established a connection between degenerate and nondegenerate problems through projection.
Provided tools for counting critical points on projective varieties.
Discussed applications to deep learning models.
Abstract
We study the critical points over an algebraic variety of an optimization problem defined by a quadratic objective that is degenerate. This scenario arises in machine learning when the dataset size is small with respect to the model, and is typically referred to as overparametrization. Our main result relates the degenerate optimization problem to a nondegenerate one via a projection. In the highly-degenerate regime, we find that a central role is played by the ramification locus of the projection. Additionally, we provide tools for counting the number of critical points over projective varieties, and discuss specific cases arising from deep learning. Our work bridges tools from algebraic geometry with ideas from machine learning, and it extends the line of literature around the Euclidean distance degree to the degenerate setting.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPolynomial and algebraic computation · Commutative Algebra and Its Applications · Tensor decomposition and applications
