Calibrated Boosting-Forest

TL;DR

Calibrated Boosting-Forest is an ensemble method that combines gradient boosting machines to deliver both high ranking power and well-calibrated probability estimates, simplifying tuning and outperforming existing models in virtual screening tasks.

Contribution

This paper introduces Calibrated Boosting-Forest, a novel ensemble technique that improves ranking and calibration in classification, reducing tuning complexity and demonstrating superior performance over deep learning.

Findings

48% improvement over deep learning models in virtual screening

95% better probability calibration than individual gradient boosting machines

Effective hyper-parameter tuning with diminishing returns

Abstract

Excellent ranking power along with well calibrated probability estimates are needed in many classification tasks. In this paper, we introduce a technique, Calibrated Boosting-Forest that captures both. This novel technique is an ensemble of gradient boosting machines that can support both continuous and binary labels. While offering superior ranking power over any individual regression or classification model, Calibrated Boosting-Forest is able to preserve well calibrated posterior probabilities. Along with these benefits, we provide an alternative to the tedious step of tuning gradient boosting machines. We demonstrate that tuning Calibrated Boosting-Forest can be reduced to a simple hyper-parameter selection. We further establish that increasing this hyper-parameter improves the ranking performance under a diminishing return. We examine the effectiveness of Calibrated Boosting-Forest on ligand-based virtual screening where both continuous and binary labels are available and compare the performance of Calibrated Boosting-Forest with logistic regression, gradient boosting machine and deep learning. Calibrated Boosting-Forest achieved an approximately 48% improvement compared to a state-of-art deep learning model. Moreover, it achieved around 95% improvement on probability quality measurement compared to the best individual gradient boosting machine. Calibrated Boosting-Forest offers a benchmark demonstration that in the field of ligand-based virtual screening, deep learning is not the universally dominant machine learning model and good calibrated probabilities can better facilitate virtual screening process.