Training Auto-encoders Effectively via Eliminating Task-irrelevant Input Variables

TL;DR

This paper introduces a method to improve auto-encoder training by removing task-irrelevant input variables, leading to better feature extraction and enhanced classification performance.

Contribution

It proposes an importance-based variable selection technique to eliminate irrelevant inputs, enhancing auto-encoder effectiveness in deep learning models.

Findings

Significant performance improvements on three datasets.

Effective removal of task-irrelevant variables enhances auto-encoder quality.

Method applicable to each layer of stacked auto-encoders.

Abstract

Auto-encoders are often used as building blocks of deep network classifier to learn feature extractors, but task-irrelevant information in the input data may lead to bad extractors and result in poor generalization performance of the network. In this paper,via dropping the task-irrelevant input variables the performance of auto-encoders can be obviously improved .Specifically, an importance-based variable selection method is proposed to aim at finding the task-irrelevant input variables and dropping them.It firstly estimates importance of each variable,and then drops the variables with importance value lower than a threshold. In order to obtain better performance, the method can be employed for each layer of stacked auto-encoders. Experimental results show that when combined with our method the stacked denoising auto-encoders achieves significantly improved performance on three challenging datasets.