Bayesian Neighbourhood Component Analysis

TL;DR

This paper introduces Bayesian NCA, a novel Bayesian approach to metric learning that improves robustness and efficiency, especially with small or noisy datasets, by modeling parameter uncertainty and leveraging a graph-based constraint system.

Contribution

The paper proposes Bayesian NCA, a new Bayesian metric learning method that uses a graph-based approach for better uncertainty modeling and robustness over traditional point-estimate methods.

Findings

Outperforms previous Bayesian metric learning methods.

Learns robust metrics from small or noisy datasets.

Demonstrates improved accuracy on multiple datasets.

Abstract

Learning a good distance metric in feature space potentially improves the performance of the KNN classifier and is useful in many real-world applications. Many metric learning algorithms are however based on the point estimation of a quadratic optimization problem, which is time-consuming, susceptible to overfitting, and lack a natural mechanism to reason with parameter uncertainty, an important property useful especially when the training set is small and/or noisy. To deal with these issues, we present a novel Bayesian metric learning method, called Bayesian NCA, based on the well-known Neighbourhood Component Analysis method, in which the metric posterior is characterized by the local label consistency constraints of observations, encoded with a similarity graph instead of independent pairwise constraints. For efficient Bayesian optimization, we explore the variational lower bound over the log-likelihood of the original NCA objective. Experiments on several publicly available datasets demonstrate that the proposed method is able to learn robust metric measures from small size dataset and/or from challenging training set with labels contaminated by errors. The proposed method is also shown to outperform a previous pairwise constrained Bayesian metric learning method.