On Cognitive Preferences and the Plausibility of Rule-based Models

TL;DR

This paper challenges the assumption that simpler rule-based models are more interpretable by examining their plausibility and user acceptance, revealing that longer explanations may sometimes be more convincing.

Contribution

It investigates the relationship between rule length and plausibility, providing empirical evidence that longer rules can be more acceptable to users, contrary to common beliefs.

Findings

No strong preference for simple rules in user judgments.

Weak preference for longer rules observed in some domains.

Cognitive biases influence perceptions of rule plausibility.

Abstract

It is conventional wisdom in machine learning and data mining that logical models such as rule sets are more interpretable than other models, and that among such rule-based models, simpler models are more interpretable than more complex ones. In this position paper, we question this latter assumption by focusing on one particular aspect of interpretability, namely the plausibility of models. Roughly speaking, we equate the plausibility of a model with the likeliness that a user accepts it as an explanation for a prediction. In particular, we argue that, all other things being equal, longer explanations may be more convincing than shorter ones, and that the predominant bias for shorter models, which is typically necessary for learning powerful discriminative models, may not be suitable when it comes to user acceptance of the learned models. To that end, we first recapitulate evidence for and against this postulate, and then report the results of an evaluation in a crowd-sourcing study based on about 3.000 judgments. The results do not reveal a strong preference for simple rules, whereas we can observe a weak preference for longer rules in some domains. We then relate these results to well-known cognitive biases such as the conjunction fallacy, the representative heuristic, or the recogition heuristic, and investigate their relation to rule length and plausibility.