Cost-Sensitive Tree of Classifiers

TL;DR

This paper introduces a cost-sensitive tree of classifiers that reduces test-time computational costs by selectively extracting features along different paths, maintaining high accuracy in large-scale applications.

Contribution

It proposes a novel tree-based classifier structure that balances accuracy and computational cost by optimizing feature extraction per input path.

Findings

Achieves high accuracy with reduced feature computation

Significantly lowers test-time costs compared to traditional methods

Effective in large-scale industrial applications

Abstract

Recently, machine learning algorithms have successfully entered large-scale real-world industrial applications (e.g. search engines and email spam filters). Here, the CPU cost during test time must be budgeted and accounted for. In this paper, we address the challenge of balancing the test-time cost and the classifier accuracy in a principled fashion. The test-time cost of a classifier is often dominated by the computation required for feature extraction-which can vary drastically across eatures. We decrease this extraction time by constructing a tree of classifiers, through which test inputs traverse along individual paths. Each path extracts different features and is optimized for a specific sub-partition of the input space. By only computing features for inputs that benefit from them the most, our cost sensitive tree of classifiers can match the high accuracies of the current state-of-the-art at a small fraction of the computational cost.