Information-theoretic Semi-supervised Metric Learning via Entropy Regularization

TL;DR

This paper introduces Seraph, an information-theoretic semi-supervised metric learning method that maximizes entropy on labeled data and minimizes it on unlabeled data, integrating supervised and unsupervised learning effectively.

Contribution

Seraph is a novel semi-supervised metric learning framework that does not depend on manifold assumptions and incorporates entropy regularization with low-rank constraints.

Findings

Seraph outperforms several existing metric learning methods.

Efficient and stable optimization via EM-like scheme.

Effective integration of labeled and unlabeled data.

Abstract

We propose a general information-theoretic approach called Seraph (SEmi-supervised metRic leArning Paradigm with Hyper-sparsity) for metric learning that does not rely upon the manifold assumption. Given the probability parameterized by a Mahalanobis distance, we maximize the entropy of that probability on labeled data and minimize it on unlabeled data following entropy regularization, which allows the supervised and unsupervised parts to be integrated in a natural and meaningful way. Furthermore, Seraph is regularized by encouraging a low-rank projection induced from the metric. The optimization of Seraph is solved efficiently and stably by an EM-like scheme with the analytical E-Step and convex M-Step. Experiments demonstrate that Seraph compares favorably with many well-known global and local metric learning methods.