Decoupling Decision-Making in Fraud Prevention through Classifier Calibration for Business Logic Action

TL;DR

This paper proposes using calibration strategies to decouple machine learning classifiers from business logic actions, improving robustness against data shifts in fraud prevention scenarios.

Contribution

It introduces a calibration-based approach for decoupling classifiers from decision actions, with a comparative analysis of calibration methods in real-world fraud detection.

Findings

Isotonic and Beta calibration methods perform well under data shift conditions.

Calibration strategies improve the robustness of classifiers in changing data environments.

Trade-offs exist between calibration methods in terms of performance and complexity.

Abstract

Machine learning models typically focus on specific targets like creating classifiers, often based on known population feature distributions in a business context. However, models calculating individual features adapt over time to improve precision, introducing the concept of decoupling: shifting from point evaluation to data distribution. We use calibration strategies as strategy for decoupling machine learning (ML) classifiers from score-based actions within business logic frameworks. To evaluate these strategies, we perform a comparative analysis using a real-world business scenario and multiple ML models. Our findings highlight the trade-offs and performance implications of the approach, offering valuable insights for practitioners seeking to optimize their decoupling efforts. In particular, the Isotonic and Beta calibration methods stand out for scenarios in which there is shift between training and testing data.