Generalized Optimal Classification Trees: A Mixed-Integer Programming Approach

TL;DR

This paper introduces a mixed-integer programming framework for learning optimal classification trees that optimize nonlinear metrics like F1-score, effectively handling class imbalance and improving scalability on real-world datasets.

Contribution

It presents a novel MIP-based approach with acceleration techniques for optimizing nonlinear performance metrics in classification trees, addressing scalability and class imbalance.

Findings

Efficiently optimizes nonlinear metrics such as F1-score.

Achieves strong predictive performance on benchmark datasets.

Reduces solution times compared to existing methods.

Abstract

Global optimization of decision trees is a long-standing challenge in combinatorial optimization, yet such models play an important role in interpretable machine learning. Although the problem has been investigated for several decades, only recent advances in discrete optimization have enabled practical algorithms for solving optimal classification tree problems on real-world datasets. Mixed-integer programming (MIP) offers a high degree of modeling flexibility, and we therefore propose a MIP-based framework for learning optimal classification trees under nonlinear performance metrics, such as the F1-score, that explicitly addresses class imbalance. To improve scalability, we develop problem-specific acceleration techniques, including a tailored branch-and-cut algorithm, an instance-reduction scheme, and warm-start strategies. We evaluate the proposed approach on 50 benchmark datasets. The computational results show that the framework can efficiently optimize nonlinear metrics while achieving strong predictive performance and reduced solution times compared with existing methods.