NUTSHELL: A Dataset for Abstract Generation from Scientific Talks

TL;DR

NUTSHELL is a new multimodal dataset pairing ACL conference talks with abstracts, designed to advance speech-to-abstract generation research in scientific communication.

Contribution

It introduces NUTSHELL, the first large-scale dataset for speech-to-abstract generation from scientific talks, along with baseline models and evaluation methods.

Findings

Strong baselines established for SAG.

Automatic and human evaluations highlight challenges and benefits.

Training on NUTSHELL improves abstract generation quality.

Abstract

Scientific communication is receiving increasing attention in natural language processing, especially to help researches access, summarize, and generate content. One emerging application in this area is Speech-to-Abstract Generation (SAG), which aims to automatically generate abstracts from recorded scientific presentations. SAG enables researchers to efficiently engage with conference talks, but progress has been limited by a lack of large-scale datasets. To address this gap, we introduce NUTSHELL, a novel multimodal dataset of *ACL conference talks paired with their corresponding abstracts. We establish strong baselines for SAG and evaluate the quality of generated abstracts using both automatic metrics and human judgments. Our results highlight the challenges of SAG and demonstrate the benefits of training on NUTSHELL. By releasing NUTSHELL under an open license (CC-BY 4.0), we aim to advance research in SAG and foster the development of improved models and evaluation methods.