A study of tree-based methods and their combination

TL;DR

This paper reviews tree-based machine learning methods, introduces a framework called ISLE for faster ensemble fitting, and proposes an adaptive combination strategy ARM along with modified ISLEs, evaluated on real datasets.

Contribution

It introduces a general ISLE framework to accelerate tree ensemble fitting and a novel ARM strategy for model combination, with performance evaluations on real data.

Findings

ISLE accelerates tree ensemble training

ARM effectively combines models for improved performance

Modified ISLEs show competitive results on datasets

Abstract

Tree-based methods are popular machine learning techniques used in various fields. In this work, we review their foundations and a general framework the importance sampled learning ensemble (ISLE) that accelerates their fitting process. Furthermore, we describe a model combination strategy called the adaptive regression by mixing (ARM), which is feasible for tree-based methods via ISLE. Moreover, three modified ISLEs are proposed, and their performance are evaluated on the real data sets.