First and Second Order Methods for Online Convolutional Dictionary Learning

TL;DR

This paper advances online convolutional dictionary learning by proposing new algorithms that improve scalability, support incomplete data, and are backed by rigorous theoretical analysis, addressing limitations of previous batch methods.

Contribution

It introduces a novel online convolutional dictionary learning algorithm with enhanced performance and the ability to incorporate spatial masks for incomplete data, along with theoretical guarantees.

Findings

New algorithms outperform previous methods in efficiency

Support for learning from incomplete data with spatial masks

Theoretical analysis confirms convergence and stability

Abstract

Convolutional sparse representations are a form of sparse representation with a structured, translation invariant dictionary. Most convolutional dictionary learning algorithms to date operate in batch mode, requiring simultaneous access to all training images during the learning process, which results in very high memory usage and severely limits the training data that can be used. Very recently, however, a number of authors have considered the design of online convolutional dictionary learning algorithms that offer far better scaling of memory and computational cost with training set size than batch methods. This paper extends our prior work, improving a number of aspects of our previous algorithm; proposing an entirely new one, with better performance, and that supports the inclusion of a spatial mask for learning from incomplete data; and providing a rigorous theoretical analysis of these methods.