ConfNet2Seq: Full Length Answer Generation from Spoken Questions

TL;DR

This paper introduces ConfNet2Seq, a novel system that generates full-length natural language answers from spoken questions represented as confusion networks, advancing spoken content understanding in dialogue systems.

Contribution

The paper presents the first method to generate full answers from confusion network representations of spoken questions, along with a large-scale dataset for training and evaluation.

Findings

Achieves comparable performance with the best ASR hypotheses

First to generate full answers from graph-based spoken content representations

Provides a large dataset of spoken questions and answers

Abstract

Conversational and task-oriented dialogue systems aim to interact with the user using natural responses through multi-modal interfaces, such as text or speech. These desired responses are in the form of full-length natural answers generated over facts retrieved from a knowledge source. While the task of generating natural answers to questions from an answer span has been widely studied, there has been little research on natural sentence generation over spoken content. We propose a novel system to generate full length natural language answers from spoken questions and factoid answers. The spoken sequence is compactly represented as a confusion network extracted from a pre-trained Automatic Speech Recognizer. This is the first attempt towards generating full-length natural answers from a graph input(confusion network) to the best of our knowledge. We release a large-scale dataset of 259,788 samples of spoken questions, their factoid answers and corresponding full-length textual answers. Following our proposed approach, we achieve comparable performance with best ASR hypothesis.