Mixed-curvature decision trees and random forests

TL;DR

This paper introduces decision trees and random forests adapted for product manifolds with heterogeneous curvature, enabling effective analysis of complex datasets in non-Euclidean spaces, and demonstrates superior performance on diverse tasks.

Contribution

It extends decision tree and random forest algorithms to product manifolds, incorporating manifold geometry and providing a novel angular reformulation that preserves algorithmic effectiveness.

Findings

Product RFs ranked first on 29 out of 57 tasks.

Method simplifies to Euclidean, hyperbolic, or hyperspherical trees in special cases.

Code is publicly available for reproducibility.

Abstract

Decision trees (DTs) and their random forest (RF) extensions are workhorses of classification and regression in Euclidean spaces. However, algorithms for learning in non-Euclidean spaces are still limited. We extend DT and RF algorithms to product manifolds: Cartesian products of several hyperbolic, hyperspherical, or Euclidean components. Such manifolds handle heterogeneous curvature while still factorizing neatly into simpler components, making them compelling embedding spaces for complex datasets. Our novel angular reformulation respects manifold geometry while preserving the algorithmic properties that make decision trees effective. In the special cases of single-component manifolds, our method simplifies to its Euclidean or hyperbolic counterparts, or introduces hyperspherical DT algorithms, depending on the curvature. In benchmarks on a diverse suite of 57 classification, regression, and link prediction tasks, our product RFs ranked first on 29 tasks and came in the top 2 for 41. This highlights the value of product RFs as straightforward yet powerful new tools for data analysis in product manifolds. Code for our method is available at https://github.com/pchlenski/manify.