DISCO: DISCovering Overfittings as Causal Rules for Text Classification Models

TL;DR

DISCO is a novel method that uncovers causal, rule-based explanations for text classification models, revealing overfitting and spurious correlations to improve interpretability and model robustness.

Contribution

It introduces a scalable sequence mining approach to discover global, causal n-gram rules that explain model predictions and detect overfitting, surpassing existing interpretability methods.

Findings

Achieved 100% detection of manually inserted shortcuts in training data.

Identified an 18.8% performance regression due to overfitting.

Enabled interactive explanations to distinguish spurious from genuine features.

Abstract

With the rapid advancement of neural language models, the deployment of over-parameterized models has surged, increasing the need for interpretable explanations comprehensible to human inspectors. Existing post-hoc interpretability methods, which often focus on unigram features of single input textual instances, fail to capture the models' decision-making process fully. Additionally, many methods do not differentiate between decisions based on spurious correlations and those based on a holistic understanding of the input. Our paper introduces DISCO, a novel method for discovering global, rule-based explanations by identifying causal n-gram associations with model predictions. This method employs a scalable sequence mining technique to extract relevant text spans from training data, associate them with model predictions, and conduct causality checks to distill robust rules that elucidate model behavior. These rules expose potential overfitting and provide insights into misleading feature combinations. We validate DISCO through extensive testing, demonstrating its superiority over existing methods in offering comprehensive insights into complex model behaviors. Our approach successfully identifies all shortcuts manually introduced into the training data (100% detection rate on the MultiRC dataset), resulting in an 18.8% regression in model performance -- a capability unmatched by any other method. Furthermore, DISCO supports interactive explanations, enabling human inspectors to distinguish spurious causes in the rule-based output. This alleviates the burden of abundant instance-wise explanations and helps assess the model's risk when encountering out-of-distribution (OOD) data.