# Linear Aggregation in Tree-based Estimators

## TL;DR

This paper introduces a new algorithm for tree-based estimators that uses linear aggregation functions, improving interpretability and performance for nonparametric regression tasks.

## Contribution

It presents a novel algorithm for axis-aligned splits with linear aggregation, along with an efficient implementation and empirical validation.

## Key findings

- Favorable performance on real-world benchmarks
- Enhanced interpretability demonstrated in a vote prediction experiment
- Open-source software package available for implementation

## Abstract

Regression trees and their ensemble methods are popular methods for nonparametric regression: they combine strong predictive performance with interpretable estimators. To improve their utility for locally smooth response surfaces, we study regression trees and random forests with linear aggregation functions. We introduce a new algorithm that finds the best axis-aligned split to fit linear aggregation functions on the corresponding nodes, and we offer a quasilinear time implementation. We demonstrate the algorithm's favorable performance on real-world benchmarks and in an extensive simulation study, and we demonstrate its improved interpretability using a large get-out-the-vote experiment. We provide an open-source software package that implements several tree-based estimators with linear aggregation functions.