Psychologically-Inspired Causal Prompts

TL;DR

This paper explores how different psychological causal assumptions in prompts affect sentiment classification in NLP, revealing the impact of underlying human processes on model responses and data interpretation.

Contribution

It introduces three psychologically-inspired causal prompts for sentiment analysis and analyzes their effects on model performance and response diversity.

Findings

Different causal prompts lead to varying model performances.

Psychological processes influence the agreement or diversity in model responses.

The study highlights the importance of causal assumptions in NLP tasks.

Abstract

NLP datasets are richer than just input-output pairs; rather, they carry causal relations between the input and output variables. In this work, we take sentiment classification as an example and look into the causal relations between the review (X) and sentiment (Y). As psychology studies show that language can affect emotion, different psychological processes are evoked when a person first makes a rating and then self-rationalizes their feeling in a review (where the sentiment causes the review, i.e., Y -> X), versus first describes their experience, and weighs the pros and cons to give a final rating (where the review causes the sentiment, i.e., X -> Y ). Furthermore, it is also a completely different psychological process if an annotator infers the original rating of the user by theory of mind (ToM) (where the review causes the rating, i.e., X -ToM-> Y ). In this paper, we verbalize these three causal mechanisms of human psychological processes of sentiment classification into three different causal prompts, and study (1) how differently they perform, and (2) what nature of sentiment classification data leads to agreement or diversity in the model responses elicited by the prompts. We suggest future work raise awareness of different causal structures in NLP tasks. Our code and data are at https://github.com/cogito233/psych-causal-prompt