Feature-Budgeted Random Forest

TL;DR

This paper introduces a novel random forest algorithm designed to minimize prediction error within a specified feature acquisition budget, balancing accuracy and cost during prediction.

Contribution

The paper presents a new random forest method that explicitly accounts for feature acquisition costs and provides theoretical guarantees on cost efficiency.

Findings

Achieves superior accuracy-cost trade-offs on benchmark datasets.

Provides near-optimal guarantees for feature acquisition costs.

Outperforms existing prediction-time algorithms in experiments.

Abstract

We seek decision rules for prediction-time cost reduction, where complete data is available for training, but during prediction-time, each feature can only be acquired for an additional cost. We propose a novel random forest algorithm to minimize prediction error for a user-specified {\it average} feature acquisition budget. While random forests yield strong generalization performance, they do not explicitly account for feature costs and furthermore require low correlation among trees, which amplifies costs. Our random forest grows trees with low acquisition cost and high strength based on greedy minimax cost-weighted-impurity splits. Theoretically, we establish near-optimal acquisition cost guarantees for our algorithm. Empirically, on a number of benchmark datasets we demonstrate superior accuracy-cost curves against state-of-the-art prediction-time algorithms.