Convex Optimization in Sums of Banach Spaces

TL;DR

This paper develops a unified convex optimization framework for reconstructing functions from linear measurements, decomposing solutions into sums of Banach space components with various regularizations, extending classical methods to abstract Banach spaces.

Contribution

It generalizes and unifies multi-kernel and sparse-dictionary learning techniques within a convex optimization approach for Banach spaces.

Findings

Provides conditions for existence of solutions.

Derives parametric representations of solutions.

Extends spline-fitting techniques to Banach spaces.

Abstract

We characterize the solution of a broad class of convex optimization problems that address the reconstruction of a function from a finite number of linear measurements. The underlying hypothesis is that the solution is decomposable as a finite sum of components, where each component belongs to its own prescribed Banach space; moreover, the problem is regularized by penalizing some composite norm of the solution. We establish general conditions for existence and derive the generic parametric representation of the solution components. These representations fall into three categories depending on the underlying regularization norm: (i) a linear expansion in terms of predefined "kernels" when the component space is a reproducing kernel Hilbert space (RKHS), (ii) a non-linear (duality) mapping of a linear combination of measurement functionals when the component Banach space is strictly convex, and, (iii) an adaptive expansion in terms of a small number of atoms within a larger dictionary when the component Banach space is not strictly convex. Our approach generalizes and unifies a number of multi-kernel (RKHS) and sparse-dictionary learning techniques for compressed sensing available in the literature. It also yields the natural extension of the classical spline-fitting techniques in (semi-)RKHS to the abstract Banach-space setting.