Newton-type Multilevel Optimization Method

TL;DR

This paper investigates the convergence properties of a multilevel optimization method inspired by second-order techniques, aiming to explain its empirical success in solving large-scale structured problems.

Contribution

It provides the first theoretical analysis linking problem structure to convergence rates of multilevel optimization algorithms.

Findings

Convergence properties of a multilevel method are characterized.

The structure of the optimization problem influences convergence rate.

The study offers a theoretical foundation for multilevel optimization performance.

Abstract

Inspired by multigrid methods for linear systems of equations, multilevel optimization methods have been proposed to solve structured optimization problems. Multilevel methods make more assumptions regarding the structure of the optimization model, and as a result, they outperform single-level methods, especially for large-scale models. The impressive performance of multilevel optimization methods is an empirical observation, and no theoretical explanation has so far been proposed. In order to address this issue, we study the convergence properties of a multilevel method that is motivated by second-order methods. We take the first step toward establishing how the structure of an optimization problem is related to the convergence rate of multilevel algorithms.