Highly Scalable and Provably Accurate Classification in Poincare Balls

TL;DR

This paper introduces scalable, provably accurate hyperbolic classifiers in Poincaré balls, enabling effective learning on hierarchical, high-dimensional data with theoretical guarantees and strong empirical performance.

Contribution

It presents the first unified framework for hyperbolic linear classifiers with provable guarantees, including a new perceptron and SVM approach tailored for Poincaré ball models.

Findings

Algorithms converge and are scalable with Euclidean complexity.

High accuracy on synthetic datasets with millions of points.

Effective on real-world datasets like single-cell RNA-seq and ImageNet.

Abstract

Many high-dimensional and large-volume data sets of practical relevance have hierarchical structures induced by trees, graphs or time series. Such data sets are hard to process in Euclidean spaces and one often seeks low-dimensional embeddings in other space forms to perform required learning tasks. For hierarchical data, the space of choice is a hyperbolic space since it guarantees low-distortion embeddings for tree-like structures. Unfortunately, the geometry of hyperbolic spaces has properties not encountered in Euclidean spaces that pose challenges when trying to rigorously analyze algorithmic solutions. Here, for the first time, we establish a unified framework for learning scalable and simple hyperbolic linear classifiers with provable performance guarantees. The gist of our approach is to focus on Poincar\'e ball models and formulate the classification problems using tangent space formalisms. Our results include a new hyperbolic and second-order perceptron algorithm as well as an efficient and highly accurate convex optimization setup for hyperbolic support vector machine classifiers. All algorithms provably converge and are highly scalable as they have complexities comparable to those of their Euclidean counterparts. Their performance accuracies on synthetic data sets comprising millions of points, as well as on complex real-world data sets such as single-cell RNA-seq expression measurements, CIFAR10, Fashion-MNIST and mini-ImageNet.