# TreeGrad: Transferring Tree Ensembles to Neural Networks

## TL;DR

This paper introduces TreeGrad, a method to convert gradient boosting decision trees into neural networks, enabling online updates and neural architecture search with minimal performance loss.

## Contribution

It presents a novel approach to transform GBDT models into neural networks, allowing online learning and architecture optimization.

## Key findings

- Minimal performance loss in conversion
- Enables online decision tree updates
- Supports neural architecture search

## Abstract

Gradient Boosting Decision Tree (GBDT) are popular machine learning algorithms with implementations such as LightGBM and in popular machine learning toolkits like Scikit-Learn. Many implementations can only produce trees in an offline manner and in a greedy manner. We explore ways to convert existing GBDT implementations to known neural network architectures with minimal performance loss in order to allow decision splits to be updated in an online manner and provide extensions to allow splits points to be altered as a neural architecture search problem. We provide learning bounds for our neural network.