A Meta-learning Formulation of the Autoencoder Problem for Non-linear Dimensionality Reduction

TL;DR

This paper introduces a meta-learning approach to reformulate autoencoders as a bi-level optimization problem, addressing their deficiencies and improving non-linear dimensionality reduction.

Contribution

It presents a novel meta-learning formulation of autoencoders as a bi-level optimization, correcting canonical autoencoder deficiencies and enhancing performance.

Findings

Reformulation as bi-level optimization improves autoencoder performance.

The new approach explicitly solves the dimensionality reduction task.

Numerical illustration demonstrates the effectiveness of the formulation.

Abstract

A rapidly growing area of research is the use of machine learning approaches such as autoencoders for dimensionality reduction of data and models in scientific applications. We show that the canonical formulation of autoencoders suffers from several deficiencies that can hinder their performance. Using a meta-learning approach, we reformulate the autoencoder problem as a bi-level optimization procedure that explicitly solves the dimensionality reduction task. We prove that the new formulation corrects the identified deficiencies with canonical autoencoders, provide a practical way to solve it, and showcase the strength of this formulation with a simple numerical illustration.