A Tree-based Dictionary Learning Framework

TL;DR

This paper introduces a hierarchical, tree-based framework for adaptive dictionary learning that organizes data into a multiscale structure, enabling structured sparse encoding and incorporation of prior knowledge.

Contribution

It presents a novel recursive clustering approach to construct dictionaries with multiscale properties, generalizing Haar wavelets and allowing prior knowledge integration.

Findings

Dictionary atoms are adaptively defined based on data clusters.

Atoms closer to the root in the tree are used more heavily in image reconstruction.

The method offers a structured, multiscale dictionary suitable for sparse encoding.

Abstract

We propose a new outline for adaptive dictionary learning methods for sparse encoding based on a hierarchical clustering of the training data. Through recursive application of a clustering method, the data is organized into a binary partition tree representing a multiscale structure. The dictionary atoms are defined adaptively based on the data clusters in the partition tree. This approach can be interpreted as a generalization of a discrete Haar wavelet transform. Furthermore, any prior knowledge on the wanted structure of the dictionary elements can be simply incorporated. The computational complexity of our proposed algorithm depends on the employed clustering method and on the chosen similarity measure between data points. Thanks to the multiscale properties of the partition tree, our dictionary is structured: when using Orthogonal Matching Pursuit to reconstruct patches from a natural image, dictionary atoms corresponding to nodes being closer to the root node in the tree have a tendency to be used with greater coefficients.