Optimization Methods for Convolutional Sparse Coding

TL;DR

This paper reviews convolutional sparse coding, discussing various optimization methods, their computational trade-offs, and demonstrating its broad applicability across different signal processing tasks.

Contribution

It provides a comprehensive overview of optimization techniques for convolutional sparse coding, highlighting their properties and suitability for various applications.

Findings

Different optimization methods vary in complexity and speed.

Convolutional sparse coding is broadly applicable across domains.

Optimization choices impact computational efficiency and boundary effects.

Abstract

Sparse and convolutional constraints form a natural prior for many optimization problems that arise from physical processes. Detecting motifs in speech and musical passages, super-resolving images, compressing videos, and reconstructing harmonic motions can all leverage redundancies introduced by convolution. Solving problems involving sparse and convolutional constraints remains a difficult computational problem, however. In this paper we present an overview of convolutional sparse coding in a consistent framework. The objective involves iteratively optimizing a convolutional least-squares term for the basis functions, followed by an L1-regularized least squares term for the sparse coefficients. We discuss a range of optimization methods for solving the convolutional sparse coding objective, and the properties that make each method suitable for different applications. In particular, we concentrate on computational complexity, speed to {\epsilon} convergence, memory usage, and the effect of implied boundary conditions. We present a broad suite of examples covering different signal and application domains to illustrate the general applicability of convolutional sparse coding, and the efficacy of the available optimization methods.