Asking the Right Question: Inferring Advice-Seeking Intentions from Personal Narratives

TL;DR

This paper introduces a new NLP task to infer the advice-seeking goal behind personal narratives, creating a large dataset and evaluating baseline models to understand social intuition and common sense in intention inference.

Contribution

The paper formulates a novel task of inferring advice-seeking intentions from narratives, and develops a large dataset with automatically extracted question pairs for training and evaluation.

Findings

The dataset contains over 20,000 narratives with paired advice questions.

Baseline models demonstrate the task's feasibility and highlight the importance of social intuition.

Human annotation shows the task relies on common sense beyond semantic understanding.

Abstract

People often share personal narratives in order to seek advice from others. To properly infer the narrator's intention, one needs to apply a certain degree of common sense and social intuition. To test the capabilities of NLP systems to recover such intuition, we introduce the new task of inferring what is the advice-seeking goal behind a personal narrative. We formulate this as a cloze test, where the goal is to identify which of two advice-seeking questions was removed from a given narrative. The main challenge in constructing this task is finding pairs of semantically plausible advice-seeking questions for given narratives. To address this challenge, we devise a method that exploits commonalities in experiences people share online to automatically extract pairs of questions that are appropriate candidates for the cloze task. This results in a dataset of over 20,000 personal narratives, each matched with a pair of related advice-seeking questions: one actually intended by the narrator, and the other one not. The dataset covers a very broad array of human experiences, from dating, to career options, to stolen iPads. We use human annotation to determine the degree to which the task relies on common sense and social intuition in addition to a semantic understanding of the narrative. By introducing several baselines for this new task we demonstrate its feasibility and identify avenues for better modeling the intention of the narrator.