Interpreting Neural Networks to Improve Politeness Comprehension

TL;DR

This paper introduces an interpretable neural network model for politeness prediction in natural language requests, revealing linguistic markers and strategies that improve understanding and performance over traditional feature-based methods.

Contribution

It presents a simple CNN-based approach that avoids manual feature engineering and provides insights into the linguistic features learned by neural networks for politeness.

Findings

Neural networks outperform feature-based models in politeness prediction.

Visualization techniques reveal subtle linguistic markers of politeness.

Adding identified politeness strategies as features improves model accuracy.

Abstract

We present an interpretable neural network approach to predicting and understanding politeness in natural language requests. Our models are based on simple convolutional neural networks directly on raw text, avoiding any manual identification of complex sentiment or syntactic features, while performing better than such feature-based models from previous work. More importantly, we use the challenging task of politeness prediction as a testbed to next present a much-needed understanding of what these successful networks are actually learning. For this, we present several network visualizations based on activation clusters, first derivative saliency, and embedding space transformations, helping us automatically identify several subtle linguistics markers of politeness theories. Further, this analysis reveals multiple novel, high-scoring politeness strategies which, when added back as new features, reduce the accuracy gap between the original featurized system and the neural model, thus providing a clear quantitative interpretation of the success of these neural networks.