Local Distance Preserving Auto-encoders using Continuous k-Nearest Neighbours Graphs

TL;DR

This paper introduces local distance preserving auto-encoders using continuous k-nearest neighbors graphs, enhancing representation learning by maintaining local data geometry and improving generative modeling.

Contribution

It proposes a novel local distance preserving loss based on continuous k-NN graphs and formulates training as a constrained optimization problem, extending to hierarchical variational auto-encoders.

Findings

Achieves state-of-the-art performance on standard datasets

Effectively preserves local data geometry in latent space

Enhances generative modeling with geometrically consistent spaces

Abstract

Auto-encoder models that preserve similarities in the data are a popular tool in representation learning. In this paper we introduce several auto-encoder models that preserve local distances when mapping from the data space to the latent space. We use a local distance preserving loss that is based on the continuous k-nearest neighbours graph which is known to capture topological features at all scales simultaneously. To improve training performance, we formulate learning as a constraint optimisation problem with local distance preservation as the main objective and reconstruction accuracy as a constraint. We generalise this approach to hierarchical variational auto-encoders thus learning generative models with geometrically consistent latent and data spaces. Our method provides state-of-the-art performance across several standard datasets and evaluation metrics.