Exact asymptotic characterisation of running time for approximate gradient descent on random graphs

TL;DR

This paper provides an exact asymptotic analysis of the running time for approximate gradient descent algorithms on random graphs, revealing they are faster than full gradient descent in certain Erdős-Rényi models.

Contribution

It introduces a probabilistic framework for the asymptotic analysis of local algorithms approximating gradient descent on random graphs, with explicit estimates of their mean running times.

Findings

Approximate gradient descent algorithms outperform full gradient descent in certain random graph models.

A probabilistic representation of running time enables precise asymptotic estimates.

Faster convergence of local algorithms on Erdős-Rényi graphs with specific connection probabilities.

Abstract

In this work we study the time complexity for the search of local minima in random graphs whose vertices have i.i.d. cost values. We show that, for Erd\"os-R\'enyi graphs with connection probability given by $\lambda/n^\alpha$ (with $\lambda > 0$ and $0 < \alpha < 1$), a family of local algorithms that approximate a gradient descent find local minima faster than the full gradient descent. Furthermore, we find a probabilistic representation for the running time of these algorithms leading to asymptotic estimates of the mean running times.