# KTBoost: Combined Kernel and Tree Boosting

## TL;DR

KTBoost is a new boosting algorithm that combines kernel and tree methods, enabling better learning of functions with mixed regularity, and it outperforms traditional boosting methods in predictive accuracy.

## Contribution

The paper introduces KTBoost, a novel ensemble method that integrates kernel and tree boosting to handle functions with varying degrees of smoothness.

## Key findings

- KTBoost outperforms both tree and kernel boosting in predictive accuracy.
- Combining kernel and tree learners captures diverse function regularities.
- Empirical results show significant improvements across multiple datasets.

## Abstract

We introduce a novel boosting algorithm called `KTBoost' which combines kernel boosting and tree boosting. In each boosting iteration, the algorithm adds either a regression tree or reproducing kernel Hilbert space (RKHS) regression function to the ensemble of base learners. Intuitively, the idea is that discontinuous trees and continuous RKHS regression functions complement each other, and that this combination allows for better learning of functions that have parts with varying degrees of regularity such as discontinuities and smooth parts. We empirically show that KTBoost significantly outperforms both tree and kernel boosting in terms of predictive accuracy in a comparison on a wide array of data sets.