Distance Shrinkage and Euclidean Embedding via Regularized Kernel Estimation

TL;DR

This paper introduces a simple, regularized kernel-based method for estimating Euclidean distance matrices from noisy data, providing theoretical guarantees and practical algorithms that scale well for large problems.

Contribution

It characterizes the distance estimate as a uniform shrinkage of observed distances, offers risk bounds for consistent estimation, and proposes an efficient scalable algorithm.

Findings

The method achieves consistent distance estimation as the number of objects grows.

The proposed algorithm scales better than traditional second order cone programming.

Numerical experiments demonstrate the practical effectiveness of the approach.

Abstract

Although recovering an Euclidean distance matrix from noisy observations is a common problem in practice, how well this could be done remains largely unknown. To fill in this void, we study a simple distance matrix estimate based upon the so-called regularized kernel estimate. We show that such an estimate can be characterized as simply applying a constant amount of shrinkage to all observed pairwise distances. This fact allows us to establish risk bounds for the estimate implying that the true distances can be estimated consistently in an average sense as the number of objects increases. In addition, such a characterization suggests an efficient algorithm to compute the distance matrix estimator, as an alternative to the usual second order cone programming known not to scale well for large problems. Numerical experiments and an application in visualizing the diversity of Vpu protein sequences from a recent HIV-1 study further demonstrate the practical merits of the proposed method.