Universality for non-linear convex variational problems

TL;DR

This paper develops a unified mathematical framework for solving non-linear convex variational problems in Banach spaces using greedy algorithms and specialized dictionaries, achieving convergence rates comparable to classical methods.

Contribution

It introduces a novel approach employing radial dictionaries and greedy algorithms for broad classes of variational problems in Banach spaces, with proven convergence rates.

Findings

Convergence rate of the method is $O(m^{-1})$.

Applicable to diverse data types including tensors and neural networks.

Framework unifies various variational problem solutions.

Abstract

This article introduces an innovative mathematical framework designed to tackle non-linear convex variational problems in reflexive Banach spaces. Our approach employs a versatile technique that can handle a broad range of variational problems, including standard ones. To carry out the process effectively, we utilize specialized sets known as radial dictionaries, where these dictionaries encompass diverse data types, such as tensors in Tucker format with bounded rank and Neural Networks with fixed architecture and bounded parameters. The core of our method lies in employing a greedy algorithm through dictionary optimization defined by a multivalued map. Significantly, our analysis shows that the convergence rate achieved by our approach is comparable to the Method of Steepest Descend implemented in a reflexive Banach space, where the convergence rate follows the order of $O(m^{-1})$.