Dynamic Metric Learning from Pairwise Comparisons

TL;DR

This paper introduces OCELAD, an adaptive online learning framework for dynamic metric learning from pairwise comparisons, effectively tracking changing data structures and outperforming existing methods.

Contribution

The paper proposes OCELAD, a novel adaptive online approach for learning and tracking evolving metrics in nonstationary environments, using an ensemble of online learners.

Findings

OCELAD outperforms previous batch and online metric learning algorithms.

The RICE ensemble effectively adapts to nonstationary data.

Significant performance improvements demonstrated on real and synthetic datasets.

Abstract

Recent work in distance metric learning has focused on learning transformations of data that best align with specified pairwise similarity and dissimilarity constraints, often supplied by a human observer. The learned transformations lead to improved retrieval, classification, and clustering algorithms due to the better adapted distance or similarity measures. Here, we address the problem of learning these transformations when the underlying constraint generation process is nonstationary. This nonstationarity can be due to changes in either the ground-truth clustering used to generate constraints or changes in the feature subspaces in which the class structure is apparent. We propose Online Convex Ensemble StrongLy Adaptive Dynamic Learning (OCELAD), a general adaptive, online approach for learning and tracking optimal metrics as they change over time that is highly robust to a variety of nonstationary behaviors in the changing metric. We apply the OCELAD framework to an ensemble of online learners. Specifically, we create a retro-initialized composite objective mirror descent (COMID) ensemble (RICE) consisting of a set of parallel COMID learners with different learning rates, demonstrate RICE-OCELAD on both real and synthetic data sets and show significant performance improvements relative to previously proposed batch and online distance metric learning algorithms.