Coulomb Autoencoders

TL;DR

This paper introduces Coulomb autoencoders, a novel generative model using Coulomb kernels with MMD, providing theoretical guarantees and demonstrating superior performance on synthetic and real datasets.

Contribution

It offers the first theoretical analysis of Coulomb kernels in MMD-based autoencoders and shows improved training convergence and generalization bounds.

Findings

Coulomb kernels improve convergence properties of MMD autoencoders.

Theoretical bounds on generalization performance are established.

Coulomb autoencoders outperform state-of-the-art models on datasets.

Abstract

Learning the true density in high-dimensional feature spaces is a well-known problem in machine learning. In this work, we consider generative autoencoders based on maximum-mean discrepancy (MMD) and provide theoretical insights. In particular, (i) we prove that MMD coupled with Coulomb kernels has optimal convergence properties, which are similar to convex functionals, thus improving the training of autoencoders, and (ii) we provide a probabilistic bound on the generalization performance, highlighting some fundamental conditions to achieve better generalization. We validate the theory on synthetic examples and on the popular dataset of celebrities' faces, showing that our model, called Coulomb autoencoders, outperform the state-of-the-art.