Improving coreference resolution with automatically predicted prosodic information

TL;DR

This paper demonstrates that automatically predicted prosodic features like pitch accents and phrasing, extracted via CNN from speech signals, can significantly enhance coreference resolution in spoken language processing.

Contribution

It introduces a CNN-based method for automatic prosodic annotation that improves coreference resolution performance on spoken language data.

Findings

Automatic prosodic features improve coreference resolution accuracy.

CNN effectively predicts pitch accents and phrase boundaries from speech.

Prosodic information enhances text-based coreference models for spoken language.

Abstract

Adding manually annotated prosodic information, specifically pitch accents and phrasing, to the typical text-based feature set for coreference resolution has previously been shown to have a positive effect on German data. Practical applications on spoken language, however, would rely on automatically predicted prosodic information. In this paper we predict pitch accents (and phrase boundaries) using a convolutional neural network (CNN) model from acoustic features extracted from the speech signal. After an assessment of the quality of these automatic prosodic annotations, we show that they also significantly improve coreference resolution.