Optimal Decision Diagrams for Classification

TL;DR

This paper introduces a new mixed-integer linear programming approach for training optimal decision diagrams, which outperform decision trees in accuracy and stability, and are more efficient to train.

Contribution

The paper presents a novel mathematical programming model for training optimal decision diagrams, extending it for fairness, parsimony, and stability, with demonstrated practical effectiveness.

Findings

ODDs can be trained in short computational times.

ODDs achieve better accuracy than optimal decision trees.

ODDs offer improved stability with minimal accuracy loss.

Abstract

Decision diagrams for classification have some notable advantages over decision trees, as their internal connections can be determined at training time and their width is not bound to grow exponentially with their depth. Accordingly, decision diagrams are usually less prone to data fragmentation in internal nodes. However, the inherent complexity of training these classifiers acted as a long-standing barrier to their widespread adoption. In this context, we study the training of optimal decision diagrams (ODDs) from a mathematical programming perspective. We introduce a novel mixed-integer linear programming model for training and demonstrate its applicability for many datasets of practical importance. Further, we show how this model can be easily extended for fairness, parsimony, and stability notions. We present numerical analyses showing that our model allows training ODDs in short computational times, and that ODDs achieve better accuracy than optimal decision trees, while allowing for improved stability without significant accuracy losses.