Intention-based Segmentation: Human Reliability and Correlation with Linguistic Cues

TL;DR

This paper investigates how human speakers segment discourse based on intentions and examines how these segments correlate with linguistic cues like noun phrases, cue words, and pauses, using a corpus of spontaneous narratives.

Contribution

It provides the first quantitative analysis of human discourse segmentation based on speaker intention and its correlation with linguistic cues.

Findings

Human segmentation is statistically reliable when based on speaker intention.

Discourse segments show significant correlation with referential noun phrases, cue words, and pauses.

Abstract

Certain spans of utterances in a discourse, referred to here as segments, are widely assumed to form coherent units. Further, the segmental structure of discourse has been claimed to constrain and be constrained by many phenomena. However, there is weak consensus on the nature of segments and the criteria for recognizing or generating them. We present quantitative results of a two part study using a corpus of spontaneous, narrative monologues. The first part evaluates the statistical reliability of human segmentation of our corpus, where speaker intention is the segmentation criterion. We then use the subjects' segmentations to evaluate the correlation of discourse segmentation with three linguistic cues (referential noun phrases, cue words, and pauses), using information retrieval metrics.