Scalable Rule Lists Learning with Sampling

TL;DR

This paper introduces a scalable sampling-based algorithm for learning nearly optimal rule lists from large datasets, providing rigorous guarantees on approximation quality and outperforming existing methods in speed and accuracy.

Contribution

The paper presents a novel sampling approach that efficiently approximates optimal rule lists with theoretical guarantees, improving scalability and quality over prior methods.

Findings

Achieves up to 100x speed-up over exact methods.

Identifies rule lists with accuracy close to the optimal.

Produces rules more similar to the optimal than heuristics.

Abstract

Learning interpretable models has become a major focus of machine learning research, given the increasing prominence of machine learning in socially important decision-making. Among interpretable models, rule lists are among the best-known and easily interpretable ones. However, finding optimal rule lists is computationally challenging, and current approaches are impractical for large datasets. We present a novel and scalable approach to learn nearly optimal rule lists from large datasets. Our algorithm uses sampling to efficiently obtain an approximation of the optimal rule list with rigorous guarantees on the quality of the approximation. In particular, our algorithm guarantees to find a rule list with accuracy very close to the optimal rule list when a rule list with high accuracy exists. Our algorithm builds on the VC-dimension of rule lists, for which we prove novel upper and lower bounds. Our experimental evaluation on large datasets shows that our algorithm identifies nearly optimal rule lists with a speed-up up to two orders of magnitude over state-of-the-art exact approaches. Moreover, our algorithm is as fast as, and sometimes faster than, recent heuristic approaches, while reporting higher quality rule lists. In addition, the rules reported by our algorithm are more similar to the rules in the optimal rule list than the rules from heuristic approaches.