The Greedy Miser: Learning under Test-time Budgets

TL;DR

The paper introduces the Greedy Miser algorithm that explicitly minimizes test-time CPU-time by considering feature extraction costs during training, improving efficiency for regression and classification tasks.

Contribution

It presents a scalable, cost-effective extension of stage-wise regression that accounts for feature extraction costs during training, optimizing test-time computational efficiency.

Findings

Outperforms prior methods in cost-effectiveness.

Scales efficiently to large datasets.

Applicable to both regression and multi-class classification.

Abstract

As machine learning algorithms enter applications in industrial settings, there is increased interest in controlling their cpu-time during testing. The cpu-time consists of the running time of the algorithm and the extraction time of the features. The latter can vary drastically when the feature set is diverse. In this paper, we propose an algorithm, the Greedy Miser, that incorporates the feature extraction cost during training to explicitly minimize the cpu-time during testing. The algorithm is a straightforward extension of stage-wise regression and is equally suitable for regression or multi-class classification. Compared to prior work, it is significantly more cost-effective and scales to larger data sets.