Linear Classifiers in Product Space Forms

TL;DR

This paper introduces novel formulations for linear classifiers on product space forms combining Euclidean, spherical, and hyperbolic spaces, proving their expressive power and demonstrating improved classification performance on complex datasets.

Contribution

It develops the first perceptron and SVM classifiers for product space forms, establishes their theoretical properties, and shows practical benefits in real-world data classification.

Findings

Linear classifiers in space forms can shatter exactly d+1 points.

Perceptron convergence is rigorously established for these classifiers.

Classification in low-dimensional product spaces improves scRNA-seq data accuracy by ~15%.

Abstract

Embedding methods for product spaces are powerful techniques for low-distortion and low-dimensional representation of complex data structures. Here, we address the new problem of linear classification in product space forms -- products of Euclidean, spherical, and hyperbolic spaces. First, we describe novel formulations for linear classifiers on a Riemannian manifold using geodesics and Riemannian metrics which generalize straight lines and inner products in vector spaces. Second, we prove that linear classifiers in $d$-dimensional space forms of any curvature have the same expressive power, i.e., they can shatter exactly $d+1$ points. Third, we formalize linear classifiers in product space forms, describe the first known perceptron and support vector machine classifiers for such spaces and establish rigorous convergence results for perceptrons. Moreover, we prove that the Vapnik-Chervonenkis dimension of linear classifiers in a product space form of dimension $d$ is \emph{at least} $d+1$. We support our theoretical findings with simulations on several datasets, including synthetic data, image data, and single-cell RNA sequencing (scRNA-seq) data. The results show that classification in low-dimensional product space forms for scRNA-seq data offers, on average, a performance improvement of $\sim15\%$ when compared to that in Euclidean spaces of the same dimension.