Sharper Convergence Rates for Nonconvex Optimisation via Reduction Mappings

TL;DR

This paper introduces a framework demonstrating how reduction mappings exploiting geometric structures in high-dimensional nonconvex problems can improve curvature, condition numbers, and convergence rates of gradient-based optimisation methods.

Contribution

It provides a theoretical analysis of how reduction mappings enhance optimisation landscapes, explaining empirical acceleration in structured nonconvex problems.

Findings

Reduction mappings improve objective curvature and conditioning.

Structured reductions lead to faster convergence rates.

The framework unifies various scenarios leveraging optimality structures.

Abstract

Many high-dimensional optimisation problems exhibit rich geometric structures in their set of minimisers, often forming smooth manifolds due to over-parametrisation or symmetries. When this structure is known, at least locally, it can be exploited through reduction mappings that reparametrise part of the parameter space to lie on the solution manifold. These reductions naturally arise from inner optimisation problems and effectively remove redundant directions, yielding a lower-dimensional objective. In this work, we introduce a general framework to understand how such reductions influence the optimisation landscape. We show that well-designed reduction mappings improve curvature properties of the objective, leading to better-conditioned problems and theoretically faster convergence for gradient-based methods. Our analysis unifies a range of scenarios where structural information at optimality is leveraged to accelerate convergence, offering a principled explanation for the empirical gains observed in such optimisation algorithms.