Representing 'how you say' with 'what you say': English corpus of focused speech and text reflecting corresponding implications

TL;DR

This paper introduces an English corpus that captures focus-related paralinguistic emphasis in speech and explores methods to encode this emphasis linguistically for improved translation that preserves implied meanings.

Contribution

It presents a novel corpus of focused speech and text, and analyzes lexical and grammatical methods to encode emphasis for better paralinguistic translation.

Findings

Corpus reflects focus placement variations in speech and text

Mapping focus involves diverse lexical and grammatical methods

Data supports future research in paralinguistic translation

Abstract

In speech communication, how something is said (paralinguistic information) is as crucial as what is said (linguistic information). As a type of paralinguistic information, English speech uses sentence stress, the heaviest prominence within a sentence, to convey emphasis. While different placements of sentence stress communicate different emphatic implications, current speech translation systems return the same translations if the utterances are linguistically identical, losing paralinguistic information. Concentrating on focus, a type of emphasis, we propose mapping paralinguistic information into the linguistic domain within the source language using lexical and grammatical devices. This method enables us to translate the paraphrased text representations instead of the transcription of the original speech and obtain translations that preserve paralinguistic information. As a first step, we present the collection of an English corpus containing speech that differed in the placement of focus along with the corresponding text, which was designed to reflect the implied meaning of the speech. Also, analyses of our corpus demonstrated that mapping of focus from the paralinguistic domain into the linguistic domain involved various lexical and grammatical methods. The data and insights from our analysis will further advance research into paralinguistic translation. The corpus will be published via LDC and our website.