Recovery of Linear Components: Reduced Complexity Autoencoder Designs

TL;DR

This paper introduces RLC, a novel autoencoder-based method that balances linear and non-linear dimensionality reduction, achieving faster training and higher accuracy than traditional techniques, demonstrated through synthetic and real-world case studies.

Contribution

The paper proposes the Recovery of Linear Components (RLC), a new autoencoder approach that improves training efficiency and accuracy over linear methods while outperforming state-of-the-art techniques in specific applications.

Findings

RLC outperforms purely linear techniques in accuracy.

RLC trains faster than standard autoencoders.

RLC surpasses current state-of-the-art in wafer measurement optimization.

Abstract

Reducing dimensionality is a key preprocessing step in many data analysis applications to address the negative effects of the curse of dimensionality and collinearity on model performance and computational complexity, to denoise the data or to reduce storage requirements. Moreover, in many applications it is desirable to reduce the input dimensions by choosing a subset of variables that best represents the entire set without any a priori information available. Unsupervised variable selection techniques provide a solution to this second problem. An autoencoder, if properly regularized, can solve both unsupervised dimensionality reduction and variable selection, but the training of large neural networks can be prohibitive in time sensitive applications. We present an approach called Recovery of Linear Components (RLC), which serves as a middle ground between linear and non-linear dimensionality reduction techniques, reducing autoencoder training times while enhancing performance over purely linear techniques. With the aid of synthetic and real world case studies, we show that the RLC, when compared with an autoencoder of similar complexity, shows higher accuracy, similar robustness to overfitting, and faster training times. Additionally, at the cost of a relatively small increase in computational complexity, RLC is shown to outperform the current state-of-the-art for a semiconductor manufacturing wafer measurement site optimization application.