Compact Rule-Based Classifier Learning via Gradient Descent

TL;DR

This paper introduces FRR, a gradient-based fuzzy rule learning system that enhances interpretability and scalability of rule-based models, achieving high accuracy with compact rule bases across diverse datasets.

Contribution

FRR is a novel gradient-based rule learning system supporting user constraints and semantic fuzzy partitions, improving interpretability and efficiency over existing neuro-fuzzy methods.

Findings

FRR outperforms traditional rule-based methods by 5% in accuracy.

FRR achieves comparable accuracy to CART with 90% fewer rules.

FRR reaches 96% of state-of-the-art accuracy with only 3% of the rule base size.

Abstract

Rule-based models are essential for high-stakes decision-making due to their transparency and interpretability, but their discrete nature creates challenges for optimization and scalability. In this work, we present the Fuzzy Rule-based Reasoner (FRR), a novel gradient-based rule learning system that supports strict user constraints over rule-based complexity while achieving competitive performance. To maximize interpretability, the FRR uses semantically meaningful fuzzy logic partitions, unattainable with existing neuro-fuzzy approaches, and sufficient (single-rule) decision-making, which avoids the combinatorial complexity of additive rule ensembles. Through extensive evaluation across 40 datasets, FRR demonstrates: (1) superior performance to traditional rule-based methods (e.g., $5\%$ average accuracy over RIPPER); (2) comparable accuracy to tree-based models (e.g., CART) using rule bases $90\%$ more compact; and (3) achieves $96\%$ of the accuracy of state-of-the-art additive rule-based models while using only sufficient rules and requiring only $3\%$ of their rule base size.