# Convolutional Dictionary Learning in Hierarchical Networks

## TL;DR

This paper introduces a hierarchical deep generative model for piecewise smooth signals, combining convolutional dictionary learning with a recursive multi-scale structure, and demonstrates its effectiveness on image data.

## Contribution

It proposes a novel hierarchical convolutional dictionary learning framework that integrates sparse coding with deep neural network structures for modeling images.

## Key findings

- Model captures multi-scale image features effectively.
- Learned features improve classification performance on MNIST.
- Algorithm efficiently alternates between coefficient estimation and filter updates.

## Abstract

Filter banks are a popular tool for the analysis of piecewise smooth signals such as natural images. Motivated by the empirically observed properties of scale and detail coefficients of images in the wavelet domain, we propose a hierarchical deep generative model of piecewise smooth signals that is a recursion across scales: the low pass scale coefficients at one layer are obtained by filtering the scale coefficients at the next layer, and adding a high pass detail innovation obtained by filtering a sparse vector. This recursion describes a linear dynamic system that is a non-Gaussian Markov process across scales and is closely related to multilayer-convolutional sparse coding (ML-CSC) generative model for deep networks, except that our model allows for deeper architectures, and combines sparse and non-sparse signal representations. We propose an alternating minimization algorithm for learning the filters in this hierarchical model given observations at layer zero, e.g., natural images. The algorithm alternates between a coefficient-estimation step and a filter update step. The coefficient update step performs sparse (detail) and smooth (scale) coding and, when unfolded, leads to a deep neural network. We use MNIST to demonstrate the representation capabilities of the model, and its derived features (coefficients) for classification.