Dictionary descent in optimization

TL;DR

This paper explores how using dictionaries instead of standard bases in convex optimization can reduce dimensionality and improve convergence rates of greedy algorithms.

Contribution

It introduces a dictionary-based approach to convex optimization, analyzing how dictionary properties influence algorithm convergence.

Findings

Dictionary methods can lower problem dimensionality

Certain dictionary properties enhance greedy algorithm convergence

The approach offers potential improvements over traditional coordinate descent

Abstract

The problem of convex optimization is studied. Usually in convex optimization the minimization is over a d-dimensional domain. Very often the convergence rate of an optimization algorithm depends on the dimension d. The algorithms studied in this paper utilize dictionaries instead of a canonical basis used in the coordinate descent algorithms. We show how this approach allows us to reduce dimensionality of the problem. Also, we investigate which properties of a dictionary are beneficial for the convergence rate of typical greedy-type algorithms.