Test cost and misclassification cost trade-off using reframing

TL;DR

This paper presents a novel approach for cost-sensitive classification that optimizes model and feature selection at deployment without retraining, using reframing techniques and quadratic approximation methods to minimize joint misclassification and test costs.

Contribution

It introduces a model-agnostic reframing approach and JROC plots for selecting feature configurations to minimize joint costs without prior knowledge or retraining.

Findings

Quadratic methods effectively approximate optimal feature configurations.

Reframing allows deployment with missing attributes to reduce joint costs.

Experimental results demonstrate improved cost minimization across various models.

Abstract

Many solutions to cost-sensitive classification (and regression) rely on some or all of the following assumptions: we have complete knowledge about the cost context at training time, we can easily re-train whenever the cost context changes, and we have technique-specific methods (such as cost-sensitive decision trees) that can take advantage of that information. In this paper we address the problem of selecting models and minimising joint cost (integrating both misclassification cost and test costs) without any of the above assumptions. We introduce methods and plots (such as the so-called JROC plots) that can work with any off-the-shelf predictive technique, including ensembles, such that we reframe the model to use the appropriate subset of attributes (the feature configuration) during deployment time. In other words, models are trained with the available attributes (once and for all) and then deployed by setting missing values on the attributes that are deemed ineffective for reducing the joint cost. As the number of feature configuration combinations grows exponentially with the number of features we introduce quadratic methods that are able to approximate the optimal configuration and model choices, as shown by the experimental results.