Relational Boosted Regression Trees

TL;DR

This paper introduces a relational adaptation of boosted regression trees that uses tensor sketch techniques to approximate residual calculations, significantly improving runtime efficiency while maintaining model quality.

Contribution

It presents a novel relational algorithm for boosted regression trees that employs tensor sketch for efficient residual sum approximation, enhancing scalability.

Findings

Achieves $(1 + \\epsilon)$-approximation of residual sums

Improves runtime complexity of boosted regression trees

Maintains comparable model accuracy

Abstract

Many tasks use data housed in relational databases to train boosted regression tree models. In this paper, we give a relational adaptation of the greedy algorithm for training boosted regression trees. For the subproblem of calculating the sum of squared residuals of the dataset, which dominates the runtime of the boosting algorithm, we provide a $(1 + \epsilon)$-approximation using the tensor sketch technique. Employing this approximation within the relational boosted regression trees algorithm leads to learning similar model parameters, but with asymptotically better runtime.