Adaptive Log-Euclidean Metrics for SPD Matrix Learning

TL;DR

This paper introduces Adaptive Log-Euclidean Metrics (ALEMs), which are learnable and adaptable metrics for SPD matrices, enhancing the performance of Riemannian neural networks by better capturing their complex geometry.

Contribution

The paper proposes ALEMs, a novel class of learnable Riemannian metrics for SPD matrices, supported by theoretical analysis and demonstrated to improve SPD neural network performance.

Findings

ALEMs outperform fixed metrics in SPD neural networks.

Theoretical analysis confirms algebraic and Riemannian properties of ALEMs.

Experimental results show improved accuracy with ALEMs on Riemannian building blocks.

Abstract

Symmetric Positive Definite (SPD) matrices have received wide attention in machine learning due to their intrinsic capacity to encode underlying structural correlation in data. Many successful Riemannian metrics have been proposed to reflect the non-Euclidean geometry of SPD manifolds. However, most existing metric tensors are fixed, which might lead to sub-optimal performance for SPD matrix learning, especially for deep SPD neural networks. To remedy this limitation, we leverage the commonly encountered pullback techniques and propose Adaptive Log-Euclidean Metrics (ALEMs), which extend the widely used Log-Euclidean Metric (LEM). Compared with the previous Riemannian metrics, our metrics contain learnable parameters, which can better adapt to the complex dynamics of Riemannian neural networks with minor extra computations. We also present a complete theoretical analysis to support our ALEMs, including algebraic and Riemannian properties. The experimental and theoretical results demonstrate the merit of the proposed metrics in improving the performance of SPD neural networks. The efficacy of our metrics is further showcased on a set of recently developed Riemannian building blocks, including Riemannian batch normalization, Riemannian Residual blocks, and Riemannian classifiers.