Minimax optimal rates for Mondrian trees and forests

TL;DR

This paper provides a theoretical analysis of Mondrian Forests, demonstrating they achieve minimax optimal rates for certain function classes, and introduces an adaptive procedure for unknown smoothness levels.

Contribution

It establishes the first minimax optimal convergence rates for Mondrian Trees and Forests in arbitrary dimensions, including an adaptive method for unknown smoothness.

Findings

Mondrian Forests achieve minimax optimal rates for s-Hölder functions.

An adaptive procedure combining Mondrian Forests with model aggregation is proposed.

Results demonstrate the theoretical soundness of Mondrian Forests in a batch setting.

Abstract

Introduced by Breiman, Random Forests are widely used classification and regression algorithms. While being initially designed as batch algorithms, several variants have been proposed to handle online learning. One particular instance of such forests is the \emph{Mondrian Forest}, whose trees are built using the so-called Mondrian process, therefore allowing to easily update their construction in a streaming fashion. In this paper, we provide a thorough theoretical study of Mondrian Forests in a batch learning setting, based on new results about Mondrian partitions. Our results include consistency and convergence rates for Mondrian Trees and Forests, that turn out to be minimax optimal on the set of $s$-H\"older function with $s \in (0,1]$ (for trees and forests) and $s \in (1,2]$ (for forests only), assuming a proper tuning of their complexity parameter in both cases. Furthermore, we prove that an adaptive procedure (to the unknown $s \in (0, 2]$) can be constructed by combining Mondrian Forests with a standard model aggregation algorithm. These results are the first demonstrating that some particular random forests achieve minimax rates \textit{in arbitrary dimension}. Owing to their remarkably simple distributional properties, which lead to minimax rates, Mondrian trees are a promising basis for more sophisticated yet theoretically sound random forests variants.