"where is this relationship going?": Understanding Relationship Trajectories in Narrative Text

TL;DR

This paper introduces a new dataset and evaluation tasks to understand how social relationships evolve in narratives, highlighting the gap between current language models and human understanding.

Contribution

It presents the Social Narrative Tree dataset and two novel tasks for modeling relationship trajectories in social narratives, advancing research in social commonsense reasoning.

Findings

Models perform significantly worse than humans on the tasks.

Current models need to go beyond surface signals to understand complex relationships.

The dataset enables new avenues for studying social relationship dynamics.

Abstract

We examine a new commonsense reasoning task: given a narrative describing a social interaction that centers on two protagonists, systems make inferences about the underlying relationship trajectory. Specifically, we propose two evaluation tasks: Relationship Outlook Prediction MCQ and Resolution Prediction MCQ. In Relationship Outlook Prediction, a system maps an interaction to a relationship outlook that captures how the interaction is expected to change the relationship. In Resolution Prediction, a system attributes a given relationship outlook to a particular resolution that explains the outcome. These two tasks parallel two real-life questions that people frequently ponder upon as they navigate different social situations: "where is this relationship going?" and "how did we end up here?". To facilitate the investigation of human social relationships through these two tasks, we construct a new dataset, Social Narrative Tree, which consists of 1250 stories documenting a variety of daily social interactions. The narratives encode a multitude of social elements that interweave to give rise to rich commonsense knowledge of how relationships evolve with respect to social interactions. We establish baseline performances using language models and the accuracies are significantly lower than human performance. The results demonstrate that models need to look beyond syntactic and semantic signals to comprehend complex human relationships.