TF Boosted Trees: A scalable TensorFlow based framework for gradient boosting

TL;DR

TF Boosted Trees is a scalable, TensorFlow-based framework for distributed gradient boosting that offers faster prediction, smaller ensembles, and improved multi-class handling through innovative architecture and regularization.

Contribution

It introduces a novel architecture and techniques for distributed training, loss differentiation, and layer-wise boosting in gradient boosted trees using TensorFlow.

Findings

Faster prediction with layer-by-layer boosting

Smaller ensembles due to novel architecture

Effective multi-class handling and regularization

Abstract

TF Boosted Trees (TFBT) is a new open-sourced frame-work for the distributed training of gradient boosted trees. It is based on TensorFlow, and its distinguishing features include a novel architecture, automatic loss differentiation, layer-by-layer boosting that results in smaller ensembles and faster prediction, principled multi-class handling, and a number of regularization techniques to prevent overfitting.