Why is "Problems" Predictive of Positive Sentiment? A Case Study of Explaining Unintuitive Features in Sentiment Classification

TL;DR

This paper investigates why certain features, like the word "problems," can be predictive of positive sentiment in sentiment analysis, and proposes methods to detect and explain such unintuitive associations to improve interpretability.

Contribution

It introduces approaches for automatically detecting and explaining unintuitive predictive features in sentiment classifiers, addressing a gap in explainable AI research.

Findings

Proposed methods effectively detect unintuitive predictive features.

Crowdsourced study confirms explanations improve user understanding.

Approaches help clarify puzzling feature-label associations.

Abstract

Explainable AI (XAI) algorithms aim to help users understand how a machine learning model makes predictions. To this end, many approaches explain which input features are most predictive of a target label. However, such explanations can still be puzzling to users (e.g., in product reviews, the word "problems" is predictive of positive sentiment). If left unexplained, puzzling explanations can have negative impacts. Explaining unintuitive associations between an input feature and a target label is an underexplored area in XAI research. We take an initial effort in this direction using unintuitive associations learned by sentiment classifiers as a case study. We propose approaches for (1) automatically detecting associations that can appear unintuitive to users and (2) generating explanations to help users understand why an unintuitive feature is predictive. Results from a crowdsourced study (N=300) found that our proposed approaches can effectively detect and explain predictive but unintuitive features in sentiment classification.