Learning Robust Features with Incremental Auto-Encoders

TL;DR

This paper introduces Incremental Auto-Encoders, a novel method for learning robust, non-linear features by iteratively denoising data on its manifold, improving classification accuracy.

Contribution

The paper proposes a new incremental auto-encoder approach that reverses a diffusion process to denoise features, enhancing robustness and hierarchical feature learning.

Findings

Improved classification performance on real-world datasets.

Effective denoising of features by reversing diffusion process.

Hierarchical feature stacking enhances robustness.

Abstract

Automatically learning features, especially robust features, has attracted much attention in the machine learning community. In this paper, we propose a new method to learn non-linear robust features by taking advantage of the data manifold structure. We first follow the commonly used trick of the trade, that is learning robust features with artificially corrupted data, which are training samples with manually injected noise. Following the idea of the auto-encoder, we first assume features should contain much information to well reconstruct the input from its corrupted copies. However, merely reconstructing clean input from its noisy copies could make data manifold in the feature space noisy. To address this problem, we propose a new method, called Incremental Auto-Encoders, to iteratively denoise the extracted features. We assume the noisy manifold structure is caused by a diffusion process. Consequently, we reverse this specific diffusion process to further contract this noisy manifold, which results in an incremental optimization of model parameters . Furthermore, we show these learned non-linear features can be stacked into a hierarchy of features. Experimental results on real-world datasets demonstrate the proposed method can achieve better classification performances.