Trainlets: Dictionary Learning in High Dimensions

TL;DR

This paper introduces a scalable online dictionary learning method for high-dimensional signals, utilizing a new multi-scale wavelet-based approach to generate large, adaptable atoms called trainlets, advancing sparse representation applications.

Contribution

It presents a novel high-dimensional dictionary learning framework using a cropped wavelet decomposition and online training, enabling efficient processing of large-scale signals beyond small image patches.

Findings

Successfully handles high-dimensional data with millions of examples.

Produces large, adaptable atoms called trainlets.

Enables multi-scale analysis with minimal border effects.

Abstract

Sparse representations has shown to be a very powerful model for real world signals, and has enabled the development of applications with notable performance. Combined with the ability to learn a dictionary from signal examples, sparsity-inspired algorithms are often achieving state-of-the-art results in a wide variety of tasks. Yet, these methods have traditionally been restricted to small dimensions mainly due to the computational constraints that the dictionary learning problem entails. In the context of image processing, this implies handling small image patches. In this work we show how to efficiently handle bigger dimensions and go beyond the small patches in sparsity-based signal and image processing methods. We build our approach based on a new cropped wavelet decomposition, which enables a multi-scale analysis with virtually no border effects. We then employ this as the base dictionary within a double sparsity model to enable the training of adaptive dictionaries. To cope with the increase of training data, while at the same time improving the training performance, we present an Online Sparse Dictionary Learning (OSDL) algorithm to train this model effectively, enabling it to handle millions of examples. This work shows that dictionary learning can be up-scaled to tackle a new level of signal dimensions, obtaining large adaptable atoms that we call trainlets.