Representation Learning via Cauchy Convolutional Sparse Coding

TL;DR

This paper introduces a novel regularisation approach for convolutional sparse coding using a Cauchy prior, leading to an iterative thresholding algorithm that improves natural image reconstruction performance over traditional methods.

Contribution

The work proposes the Iterative Cauchy Thresholding algorithm for CSC, utilizing a Cauchy prior for enhanced sparsity enforcement in feature map coefficients.

Findings

ICT outperforms IHT and IST in image reconstruction tasks.

Average PSNR improvements of up to 11.30 over ISTA and 7.04 over IHT.

Demonstrates effectiveness of Cauchy prior in sparse coding.

Abstract

In representation learning, Convolutional Sparse Coding (CSC) enables unsupervised learning of features by jointly optimising both an \(\ell_2\)-norm fidelity term and a sparsity enforcing penalty. This work investigates using a regularisation term derived from an assumed Cauchy prior for the coefficients of the feature maps of a CSC generative model. The sparsity penalty term resulting from this prior is solved via its proximal operator, which is then applied iteratively, element-wise, on the coefficients of the feature maps to optimise the CSC cost function. The performance of the proposed Iterative Cauchy Thresholding (ICT) algorithm in reconstructing natural images is compared against the common choice of \(\ell_1\)-norm optimised via soft and hard thresholding. ICT outperforms IHT and IST in most of these reconstruction experiments across various datasets, with an average PSNR of up to 11.30 and 7.04 above ISTA and IHT respectively.