Pedagogical Rule Extraction to Learn Interpretable Models - an Empirical Study

TL;DR

This paper presents an empirical study on pedagogical rule extraction methods, demonstrating how they can improve interpretable models in small-data domains like medicine, through a unified framework and extensive experiments.

Contribution

It introduces the PRELIM framework to unify existing pedagogical rule extraction techniques and identifies promising configurations through comprehensive experiments.

Findings

Certain PRELIM configurations outperform existing methods

Pedagogical rule extraction enhances interpretability with small datasets

The framework facilitates systematic comparison of rule extraction techniques

Abstract

Machine-learning models are ubiquitous. In some domains, for instance, in medicine, the models' predictions must be interpretable. Decision trees, classification rules, and subgroup discovery are three broad categories of supervised machine-learning models presenting knowledge in the form of interpretable rules. The accuracy of these models learned from small datasets is usually low. Obtaining larger datasets is often hard to impossible. Pedagogical rule extraction methods could help to learn better rules from small data by augmenting a dataset employing statistical models and using it to learn a rule-based model. However, existing evaluation of these methods is often inconclusive, and they were not compared so far. Our framework PRELIM unifies existing pedagogical rule extraction techniques. In the extensive experiments, we identified promising PRELIM configurations not studied before.