Deep Kernelized Autoencoders

TL;DR

This paper introduces a deep autoencoder model that explicitly maps inputs to a user-defined kernel space and back, optimizing both reconstruction and kernel alignment for improved representation learning.

Contribution

It presents a novel deep autoencoder architecture that incorporates kernel alignment, enabling explicit control over learned representations and emulating kernel PCA.

Findings

Effective reconstruction of input data.

Successful kernel alignment with prior kernel matrices.

Promising results in denoising tasks using kernelized autoencoders.

Abstract

In this paper we introduce the deep kernelized autoencoder, a neural network model that allows an explicit approximation of (i) the mapping from an input space to an arbitrary, user-specified kernel space and (ii) the back-projection from such a kernel space to input space. The proposed method is based on traditional autoencoders and is trained through a new unsupervised loss function. During training, we optimize both the reconstruction accuracy of input samples and the alignment between a kernel matrix given as prior and the inner products of the hidden representations computed by the autoencoder. Kernel alignment provides control over the hidden representation learned by the autoencoder. Experiments have been performed to evaluate both reconstruction and kernel alignment performance. Additionally, we applied our method to emulate kPCA on a denoising task obtaining promising results.