An Additive Autoencoder for Dimension Estimation

TL;DR

This paper introduces an additive autoencoder architecture for estimating the intrinsic dimension of datasets, demonstrating that deeper networks improve autoencoding accuracy without changing the estimated dimension.

Contribution

The paper proposes a novel additive autoencoder structure and compares shallow versus deep networks for intrinsic dimension estimation.

Findings

Deeper networks achieve lower autoencoding errors.

The estimated intrinsic dimension remains consistent across shallow and deep networks.

Shallow networks are sufficient for accurate dimension detection.

Abstract

An additive autoencoder for dimension reduction, which is composed of a serially performed bias estimation, linear trend estimation, and nonlinear residual estimation, is proposed and analyzed. Computational experiments confirm that an autoencoder of this form, with only a shallow network to encapsulate the nonlinear behavior, is able to identify an intrinsic dimension of a dataset with a low autoencoding error. This observation leads to an investigation in which shallow and deep network structures, and how they are trained, are compared. We conclude that the deeper network structures obtain lower autoencoding errors during the identification of the intrinsic dimension. However, the detected dimension does not change compared to a shallow network.