Learning a Deep Part-based Representation by Preserving Data Distribution

TL;DR

This paper introduces DPNE, a deep autoencoder-based method that preserves data distribution to learn a low-dimensional, part-based representation, effectively maintaining the intrinsic structure of high-dimensional data.

Contribution

The paper proposes a novel distribution-preserving deep autoencoder method called DPNE that captures data distribution and structure in low-dimensional space.

Findings

DPNE achieves high cluster accuracy and AMI on real-world datasets.

The method effectively preserves the manifold structure in low-dimensional representations.

Abstract

Unsupervised dimensionality reduction is one of the commonly used techniques in the field of high dimensional data recognition problems. The deep autoencoder network which constrains the weights to be non-negative, can learn a low dimensional part-based representation of data. On the other hand, the inherent structure of the each data cluster can be described by the distribution of the intraclass samples. Then one hopes to learn a new low dimensional representation which can preserve the intrinsic structure embedded in the original high dimensional data space perfectly. In this paper, by preserving the data distribution, a deep part-based representation can be learned, and the novel algorithm is called Distribution Preserving Network Embedding (DPNE). In DPNE, we first need to estimate the distribution of the original high dimensional data using the $k$-nearest neighbor kernel density estimation, and then we seek a part-based representation which respects the above distribution. The experimental results on the real-world data sets show that the proposed algorithm has good performance in terms of cluster accuracy and AMI. It turns out that the manifold structure in the raw data can be well preserved in the low dimensional feature space.