Optimizing Binary Decision Diagrams with MaxSAT for classification

TL;DR

This paper introduces a novel MaxSAT-based approach to optimize Binary Decision Diagrams for classification tasks, improving interpretability and prediction accuracy over existing methods.

Contribution

It presents the first MaxSAT-based models for learning optimal BDDs with limited depth, and introduces a subtree merging technique to reduce fragmentation.

Findings

Enhanced prediction quality compared to state-of-the-art methods.

More interpretable models with lighter size.

Empirical results demonstrate clear benefits in accuracy and interpretability.

Abstract

The growing interest in explainable artificial intelligence (XAI) for critical decision making motivates the need for interpretable machine learning (ML) models. In fact, due to their structure (especially with small sizes), these models are inherently understandable by humans. Recently, several exact methods for computing such models are proposed to overcome weaknesses of traditional heuristic methods by providing more compact models or better prediction quality. Despite their compressed representation of Boolean functions, Binary decision diagrams (BDDs) did not gain enough interest as other interpretable ML models. In this paper, we first propose SAT-based models for learning optimal BDDs (in terms of the number of features) that classify all input examples. Then, we lift the encoding to a MaxSAT model to learn optimal BDDs in limited depths, that maximize the number of examples correctly classified. Finally, we tackle the fragmentation problem by introducing a method to merge compatible subtrees for the BDDs found via the MaxSAT model. Our empirical study shows clear benefits of the proposed approach in terms of prediction quality and intrepretability (i.e., lighter size) compared to the state-of-the-art approaches.