Sparse Factorization Layers for Neural Networks with Limited Supervision

TL;DR

This paper introduces two novel dictionary learning-based layers for neural networks that enhance performance under limited supervision by enabling semi-supervised learning and outperforming traditional CNNs in low-data regimes.

Contribution

The paper proposes sparse factorization layers for neural networks, integrating dictionary learning into CNNs for improved semi-supervised learning capabilities.

Findings

Outperform classical CNNs with limited labeled data.

Show performance improvements with same number of parameters.

Enable end-to-end training with back-propagation.

Abstract

Whereas CNNs have demonstrated immense progress in many vision problems, they suffer from a dependence on monumental amounts of labeled training data. On the other hand, dictionary learning does not scale to the size of problems that CNNs can handle, despite being very effective at low-level vision tasks such as denoising and inpainting. Recently, interest has grown in adapting dictionary learning methods for supervised tasks such as classification and inverse problems. We propose two new network layers that are based on dictionary learning: a sparse factorization layer and a convolutional sparse factorization layer, analogous to fully-connected and convolutional layers, respectively. Using our derivations, these layers can be dropped in to existing CNNs, trained together in an end-to-end fashion with back-propagation, and leverage semisupervision in ways classical CNNs cannot. We experimentally compare networks with these two new layers against a baseline CNN. Our results demonstrate that networks with either of the sparse factorization layers are able to outperform classical CNNs when supervised data are few. They also show performance improvements in certain tasks when compared to the CNN with no sparse factorization layers with the same exact number of parameters.