A Dataset and Benchmark for Consumer Healthcare Question Summarization

TL;DR

This paper introduces CHQ-Sum, a new domain-expert annotated dataset of consumer health questions and summaries, to advance healthcare question summarization research.

Contribution

It provides the first large-scale, domain-expert annotated dataset for consumer healthcare question summarization, enabling better model development.

Findings

State-of-the-art models perform variably on the dataset

The dataset improves understanding of consumer health questions

Benchmark results highlight future research directions

Abstract

The quest for seeking health information has swamped the web with consumers health-related questions. Generally, consumers use overly descriptive and peripheral information to express their medical condition or other healthcare needs, contributing to the challenges of natural language understanding. One way to address this challenge is to summarize the questions and distill the key information of the original question. Recently, large-scale datasets have significantly propelled the development of several summarization tasks, such as multi-document summarization and dialogue summarization. However, a lack of a domain-expert annotated dataset for the consumer healthcare questions summarization task inhibits the development of an efficient summarization system. To address this issue, we introduce a new dataset, CHQ-Sum,m that contains 1507 domain-expert annotated consumer health questions and corresponding summaries. The dataset is derived from the community question answering forum and therefore provides a valuable resource for understanding consumer health-related posts on social media. We benchmark the dataset on multiple state-of-the-art summarization models to show the effectiveness of the dataset