Deep Learning of Part-based Representation of Data Using Sparse Autoencoders with Nonnegativity Constraints

TL;DR

This paper introduces a novel deep autoencoder with nonnegativity constraints that learns part-based data representations, enhancing sparsity, reconstruction quality, and prediction performance across image and text datasets.

Contribution

The paper proposes a new nonnegativity constraint algorithm for deep autoencoders that promotes part-based feature learning and improves overall data representation.

Findings

Part-based features learned by NCAE improve interpretability.

NCAE outperforms traditional sparse autoencoders in reconstruction quality.

Representation learned by NCAE enhances prediction accuracy.

Abstract

We demonstrate a new deep learning autoencoder network, trained by a nonnegativity constraint algorithm (NCAE), that learns features which show part-based representation of data. The learning algorithm is based on constraining negative weights. The performance of the algorithm is assessed based on decomposing data into parts and its prediction performance is tested on three standard image data sets and one text dataset. The results indicate that the nonnegativity constraint forces the autoencoder to learn features that amount to a part-based representation of data, while improving sparsity and reconstruction quality in comparison with the traditional sparse autoencoder and Nonnegative Matrix Factorization. It is also shown that this newly acquired representation improves the prediction performance of a deep neural network.