Sparse Bayesian Dictionary Learning with a Gaussian Hierarchical Model

TL;DR

This paper introduces a hierarchical Bayesian approach for dictionary learning that effectively promotes sparsity and accurately infers model parameters without prior noise knowledge, outperforming existing methods especially with limited data.

Contribution

It proposes a novel Gaussian-inverse Gamma hierarchical Bayesian model and inference algorithms for dictionary learning, improving accuracy and robustness over previous techniques.

Findings

Better dictionary learning accuracy than existing methods

Effective sparsity promotion via hierarchical Bayesian priors

Robust inference without prior noise variance knowledge

Abstract

We consider a dictionary learning problem whose objective is to design a dictionary such that the signals admits a sparse or an approximate sparse representation over the learned dictionary. Such a problem finds a variety of applications such as image denoising, feature extraction, etc. In this paper, we propose a new hierarchical Bayesian model for dictionary learning, in which a Gaussian-inverse Gamma hierarchical prior is used to promote the sparsity of the representation. Suitable priors are also placed on the dictionary and the noise variance such that they can be reasonably inferred from the data. Based on the hierarchical model, a variational Bayesian method and a Gibbs sampling method are developed for Bayesian inference. The proposed methods have the advantage that they do not require the knowledge of the noise variance \emph{a priori}. Numerical results show that the proposed methods are able to learn the dictionary with an accuracy better than existing methods, particularly for the case where there is a limited number of training signals.