TrIM: Transformed Iterative Mondrian Forests for Gradient-based Dimension Reduction and High-Dimensional Regression

TL;DR

This paper introduces TrIM, an efficient algorithm that combines Mondrian forests with iterative gradient-based updates to identify relevant feature subspaces for high-dimensional regression, with proven consistency and demonstrated effectiveness.

Contribution

The paper presents the TrIM algorithm, integrating iterative updates with Mondrian forests for improved gradient-based dimension reduction in high-dimensional settings.

Findings

TrIM effectively identifies relevant feature subspaces in high-dimensional data.

The algorithm achieves consistency and favorable convergence rates.

Empirical results show improved performance on simulated and real datasets.

Abstract

We propose a computationally efficient algorithm for gradient-based linear dimension reduction and high-dimensional regression. The algorithm initially computes a Mondrian forest and uses this estimator to identify a relevant feature subspace of the inputs from an estimate of the expected gradient outer product (EGOP) of the regression function. In addition, we introduce an iterative approach known as Transformed Iterative Mondrian (TrIM) forest to improve the Mondrian forest estimator by using the EGOP estimate to update the set of features and weights used by the Mondrian partitioning mechanism. We obtain consistency guarantees and convergence rates for the estimation of the EGOP matrix and the random forest estimator obtained from one iteration of the TrIM algorithm. Lastly, we demonstrate the effectiveness of our proposed algorithm for learning the relevant feature subspace across a variety of settings with both simulated and real data.