Towards Data Distillation for End-to-end Spoken Conversational Question Answering

TL;DR

This paper introduces a new task, SCQA, for conversational question answering from speech and text, and proposes DDNet, a data distillation method that fuses audio-text features to improve system performance.

Contribution

The paper presents SCQA, a novel conversational QA task from speech, and DDNet, a unified data distillation approach that directly fuses audio and text features for better accuracy.

Findings

DDNet outperforms baseline models in spoken conversational QA.

The Spoken-CoQA dataset contains over 120,000 question-answer pairs.

Direct audio-text feature fusion reduces misalignment issues in speech QA.

Abstract

In spoken question answering, QA systems are designed to answer questions from contiguous text spans within the related speech transcripts. However, the most natural way that human seek or test their knowledge is via human conversations. Therefore, we propose a new Spoken Conversational Question Answering task (SCQA), aiming at enabling QA systems to model complex dialogues flow given the speech utterances and text corpora. In this task, our main objective is to build a QA system to deal with conversational questions both in spoken and text forms, and to explore the plausibility of providing more cues in spoken documents with systems in information gathering. To this end, instead of adopting automatically generated speech transcripts with highly noisy data, we propose a novel unified data distillation approach, DDNet, which directly fuse audio-text features to reduce the misalignment between automatic speech recognition hypotheses and the reference transcriptions. In addition, to evaluate the capacity of QA systems in a dialogue-style interaction, we assemble a Spoken Conversational Question Answering (Spoken-CoQA) dataset with more than 120k question-answer pairs. Experiments demonstrate that our proposed method achieves superior performance in spoken conversational question answering.