Multiple Scale Methods For Optimization Of Discretized Continuous Functions

TL;DR

This paper introduces a multiscale optimization framework for Lipschitz continuous functions that improves efficiency and accuracy over single-scale methods, supported by theoretical guarantees and practical experiments.

Contribution

It develops a multiscale approach with convergence guarantees for optimizing discretized continuous functions, enhancing speed and accuracy.

Findings

Achieves tighter error bounds than single-scale methods

Provides provable convergence guarantees for the multiscale approach

Demonstrates an order of magnitude speedup in probability density estimation tasks

Abstract

A multiscale optimization framework for problems over a space of Lipschitz continuous functions is developed. The method solves a coarse-grid discretization followed by linear interpolation to warm-start project gradient descent on progressively finer grids. Greedy and lazy variants are analyzed and convergence guarantees are derived that show the multiscale approach achieves provably tighter error bounds at lower computational cost than single-scale optimization. The analysis extends to any base algorithm with iterate convergence at a fixed rate. Constraint modification techniques preserve feasibility across scales. Numerical experiments on probability density estimation problems, including geological data, demonstrate speedups of an order of magnitude or better.