Big Data Regression Using Tree Based Segmentation

TL;DR

This paper introduces a scalable two-step regression method for large datasets, combining tree-based segmentation with segment-specific models, achieving predictive performance comparable to ensemble methods while maintaining interpretability.

Contribution

The paper presents a novel two-step regression approach that scales to large datasets by segmenting data with trees and applying tailored models to each segment.

Findings

Predictive performance matches Gradient Boosted Trees.

Method offers interpretability alongside scalability.

Effective for large-scale regression problems.

Abstract

Scaling regression to large datasets is a common problem in many application areas. We propose a two step approach to scaling regression to large datasets. Using a regression tree (CART) to segment the large dataset constitutes the first step of this approach. The second step of this approach is to develop a suitable regression model for each segment. Since segment sizes are not very large, we have the ability to apply sophisticated regression techniques if required. A nice feature of this two step approach is that it can yield models that have good explanatory power as well as good predictive performance. Ensemble methods like Gradient Boosted Trees can offer excellent predictive performance but may not provide interpretable models. In the experiments reported in this study, we found that the predictive performance of the proposed approach matched the predictive performance of Gradient Boosted Trees.