Convex Relaxation for Solving Large-Margin Classifiers in Hyperbolic Space

TL;DR

This paper introduces a convex relaxation approach for large-margin classifiers in hyperbolic space, addressing the non-convexity challenge and outperforming gradient descent methods in empirical tests.

Contribution

It reformulates hyperbolic SVMs as polynomial optimization problems and applies semidefinite relaxation techniques for better solutions.

Findings

Semidefinite relaxation outperforms projected gradient descent.

Proposed methods achieve higher accuracy in hyperbolic SVMs.

Relaxation techniques effectively handle non-convex hyperbolic optimization.

Abstract

Hyperbolic spaces have increasingly been recognized for their outstanding performance in handling data with inherent hierarchical structures compared to their Euclidean counterparts. However, learning in hyperbolic spaces poses significant challenges. In particular, extending support vector machines to hyperbolic spaces is in general a constrained non-convex optimization problem. Previous and popular attempts to solve hyperbolic SVMs, primarily using projected gradient descent, are generally sensitive to hyperparameters and initializations, often leading to suboptimal solutions. In this work, by first rewriting the problem into a polynomial optimization, we apply semidefinite relaxation and sparse moment-sum-of-squares relaxation to effectively approximate the optima. From extensive empirical experiments, these methods are shown to perform better than the projected gradient descent approach.