Contractive De-noising Auto-encoder

TL;DR

This paper introduces a novel auto-encoder, called contractive de-noising auto-encoder (CDAE), which combines the robustness features of both de-noising auto-encoders and contractive auto-encoders to improve feature learning and classification performance.

Contribution

The paper proposes a new auto-encoder model that integrates de-noising and contractive techniques, enhancing robustness and feature extraction capabilities.

Findings

CDAE outperforms DAE and CAE on MNIST dataset.

Stacked CDAE extracts more abstract features.

CDAE improves classification accuracy with SVM.

Abstract

Auto-encoder is a special kind of neural network based on reconstruction. De-noising auto-encoder (DAE) is an improved auto-encoder which is robust to the input by corrupting the original data first and then reconstructing the original input by minimizing the reconstruction error function. And contractive auto-encoder (CAE) is another kind of improved auto-encoder to learn robust feature by introducing the Frobenius norm of the Jacobean matrix of the learned feature with respect to the original input. In this paper, we combine de-noising auto-encoder and contractive auto- encoder, and propose another improved auto-encoder, contractive de-noising auto- encoder (CDAE), which is robust to both the original input and the learned feature. We stack CDAE to extract more abstract features and apply SVM for classification. The experiment result on benchmark dataset MNIST shows that our proposed CDAE performed better than both DAE and CAE, proving the effective of our method.