Gradient Regularized Budgeted Boosting

TL;DR

This paper introduces a semi-supervised gradient boosted regression trees algorithm that effectively incorporates unlabeled data to learn cost-efficient classifiers under budget constraints, addressing the scarcity of labeled data in real-world applications.

Contribution

It presents the first semi-supervised approach for budgeted learning using gradient boosted trees, leveraging unlabeled data via Laplace smoothing.

Findings

Successfully incorporates unlabeled data into budgeted learning

Maintains classifier performance within test-time cost constraints

First semi-supervised method for budgeted gradient boosting

Abstract

As machine learning transitions increasingly towards real world applications controlling the test-time cost of algorithms becomes more and more crucial. Recent work, such as the Greedy Miser and Speedboost, incorporate test-time budget constraints into the training procedure and learn classifiers that provably stay within budget (in expectation). However, so far, these algorithms are limited to the supervised learning scenario where sufficient amounts of labeled data are available. In this paper we investigate the common scenario where labeled data is scarce but unlabeled data is available in abundance. We propose an algorithm that leverages the unlabeled data (through Laplace smoothing) and learns classifiers with budget constraints. Our model, based on gradient boosted regression trees (GBRT), is, to our knowledge, the first algorithm for semi-supervised budgeted learning.