# Convolutional dictionary learning based auto-encoders for natural exponential-family distributions

## TL;DR

This paper presents a novel auto-encoder framework based on convolutional dictionary learning tailored for natural exponential-family data, combining generative models with deep learning for improved unsupervised and supervised tasks.

## Contribution

It introduces a scalable neural network architecture that integrates convolutional dictionary learning with exponential-family distributions, enabling flexible unsupervised and supervised learning for non-Gaussian data.

## Key findings

- Better fit for neural spiking data than baselines
- Competitive Poisson image denoising with fewer parameters
- Analysis of gradient dynamics in binomial auto-encoders

## Abstract

We introduce a class of auto-encoder neural networks tailored to data from the natural exponential family (e.g., count data). The architectures are inspired by the problem of learning the filters in a convolutional generative model with sparsity constraints, often referred to as convolutional dictionary learning (CDL). Our work is the first to combine ideas from convolutional generative models and deep learning for data that are naturally modeled with a non-Gaussian distribution (e.g., binomial and Poisson). This perspective provides us with a scalable and flexible framework that can be re-purposed for a wide range of tasks and assumptions on the generative model. Specifically, the iterative optimization procedure for solving CDL, an unsupervised task, is mapped to an unfolded and constrained neural network, with iterative adjustments to the inputs to account for the generative distribution. We also show that the framework can easily be extended for discriminative training, appropriate for a supervised task. We demonstrate 1) that fitting the generative model to learn, in an unsupervised fashion, the latent stimulus that underlies neural spiking data leads to better goodness-of-fit compared to other baselines, 2) competitive performance compared to state-of-the-art algorithms for supervised Poisson image denoising, with significantly fewer parameters, and 3) gradient dynamics of shallow binomial auto-encoder.