Metric Learning in an RKHS

TL;DR

This paper develops a theoretical framework for nonlinear metric learning in RKHS, providing generalization guarantees and sample complexity bounds, supported by simulations and real data experiments.

Contribution

It introduces a general RKHS framework for metric learning and offers the first theoretical analysis with guarantees for nonlinear methods.

Findings

Established generalization bounds for RKHS metric learning.

Derived sample complexity bounds for the proposed methods.

Validated theoretical results through simulations and real data experiments.

Abstract

Metric learning from a set of triplet comparisons in the form of "Do you think item h is more similar to item i or item j?", indicating similarity and differences between items, plays a key role in various applications including image retrieval, recommendation systems, and cognitive psychology. The goal is to learn a metric in the RKHS that reflects the comparisons. Nonlinear metric learning using kernel methods and neural networks have shown great empirical promise. While previous works have addressed certain aspects of this problem, there is little or no theoretical understanding of such methods. The exception is the special (linear) case in which the RKHS is the standard Euclidean space $\mathbb{R}^d$; there is a comprehensive theory for metric learning in $\mathbb{R}^d$. This paper develops a general RKHS framework for metric learning and provides novel generalization guarantees and sample complexity bounds. We validate our findings through a set of simulations and experiments on real datasets. Our code is publicly available at https://github.com/RamyaLab/metric-learning-RKHS.