Interpretable Two-level Boolean Rule Learning for Classification

TL;DR

This paper introduces a new optimization framework for learning accurate, sparse, and interpretable two-level Boolean rules in both CNF and DNF forms, balancing accuracy and simplicity.

Contribution

It presents a novel LP-based optimization approach for two-level Boolean rule learning, enhancing interpretability without sacrificing accuracy.

Findings

Effective tradeoff between accuracy and interpretability.

Algorithms outperform existing methods in sparse rule learning.

Flexible framework applicable to CNF and DNF forms.

Abstract

As a contribution to interpretable machine learning research, we develop a novel optimization framework for learning accurate and sparse two-level Boolean rules. We consider rules in both conjunctive normal form (AND-of-ORs) and disjunctive normal form (OR-of-ANDs). A principled objective function is proposed to trade classification accuracy and interpretability, where we use Hamming loss to characterize accuracy and sparsity to characterize interpretability. We propose efficient procedures to optimize these objectives based on linear programming (LP) relaxation, block coordinate descent, and alternating minimization. Experiments show that our new algorithms provide very good tradeoffs between accuracy and interpretability.