OTLP: Output Thresholding Using Mixed Integer Linear Programming

TL;DR

OTLP is a versatile, model-agnostic thresholding framework using mixed integer linear programming, designed to optimize classifier thresholds especially in imbalanced classification scenarios, demonstrated on credit card fraud detection.

Contribution

Introduces OTLP, a novel thresholding method that supports diverse objectives and constraints, addressing limitations of existing theoretical techniques in complex real-world applications.

Findings

Effective in imbalanced classification tasks

Supports various objective functions and constraints

Demonstrated on credit card fraud detection dataset

Abstract

Output thresholding is the technique to search for the best threshold to be used during inference for any classifiers that can produce probability estimates on train and testing datasets. It is particularly useful in high imbalance classification problems where the default threshold is not able to refer to imbalance in class distributions and fail to give the best performance. This paper proposes OTLP, a thresholding framework using mixed integer linear programming which is model agnostic, can support different objective functions and different set of constraints for a diverse set of problems including both balanced and imbalanced classification problems. It is particularly useful in real world applications where the theoretical thresholding techniques are not able to address to product related requirements and complexity of the applications which utilize machine learning models. Through the use of Credit Card Fraud Detection Dataset, we evaluate the usefulness of the framework.