Help! Need Advice on Identifying Advice

TL;DR

This paper introduces a dataset from Reddit advice forums annotated for advice sentences, analyzes linguistic features, and evaluates models for advice identification, highlighting challenges and future directions.

Contribution

It provides a new annotated dataset and analysis for advice detection in online forums, along with preliminary model evaluations and insights.

Findings

Pre-trained language models outperform rule-based systems in advice detection.

Advice identification remains challenging despite model improvements.

Rich linguistic phenomena characterize advice discourse.

Abstract

Humans use language to accomplish a wide variety of tasks - asking for and giving advice being one of them. In online advice forums, advice is mixed in with non-advice, like emotional support, and is sometimes stated explicitly, sometimes implicitly. Understanding the language of advice would equip systems with a better grasp of language pragmatics; practically, the ability to identify advice would drastically increase the efficiency of advice-seeking online, as well as advice-giving in natural language generation systems. We present a dataset in English from two Reddit advice forums - r/AskParents and r/needadvice - annotated for whether sentences in posts contain advice or not. Our analysis reveals rich linguistic phenomena in advice discourse. We present preliminary models showing that while pre-trained language models are able to capture advice better than rule-based systems, advice identification is challenging, and we identify directions for future research. Comments: To be presented at EMNLP 2020.