Graph Metric Learning via Gershgorin Disc Alignment

TL;DR

This paper introduces a fast, projection-free graph metric learning framework that leverages Gershgorin disc alignment for efficient optimization within the set of generalized graph Laplacian matrices, outperforming existing methods.

Contribution

The paper presents a novel optimization approach using Gershgorin disc alignment and eigenvector updates for efficient metric learning on graph Laplacians, including positive-diagonal matrices.

Findings

Outperforms competing metric learning methods in classification tasks.

Efficient linear programming solution via Frank-Wolfe iterations.

Effective eigenvector updates using LOBPCG with warm start.

Abstract

We propose a fast general projection-free metric learning framework, where the minimization objective $\min_{\textbf{M} \in \mathcal{S}} Q(\textbf{M})$ is a convex differentiable function of the metric matrix $\textbf{M}$, and $\textbf{M}$ resides in the set $\mathcal{S}$ of generalized graph Laplacian matrices for connected graphs with positive edge weights and node degrees. Unlike low-rank metric matrices common in the literature, $\mathcal{S}$ includes the important positive-diagonal-only matrices as a special case in the limit. The key idea for fast optimization is to rewrite the positive definite cone constraint in $\mathcal{S}$ as signal-adaptive linear constraints via Gershgorin disc alignment, so that the alternating optimization of the diagonal and off-diagonal terms in $\textbf{M}$ can be solved efficiently as linear programs via Frank-Wolfe iterations. We prove that the Gershgorin discs can be aligned perfectly using the first eigenvector $\textbf{v}$ of $\textbf{M}$, which we update iteratively using Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) with warm start as diagonal / off-diagonal terms are optimized. Experiments show that our efficiently computed graph metric matrices outperform metrics learned using competing methods in terms of classification tasks.