Re-framing Incremental Deep Language Models for Dialogue Processing with Multi-task Learning

TL;DR

This paper introduces a multi-task learning framework for incremental dialogue processing that combines disfluency detection, language modeling, POS tagging, and utterance segmentation in a deep recurrent model, improving performance and aiding conversational agents.

Contribution

The paper proposes a novel multi-task learning approach that effectively integrates four dialogue processing tasks into a single incremental model, demonstrating improved performance and potential for psychiatric conversational agents.

Findings

Multi-task learning enhances dialogue processing performance.

The model outperforms individual task models.

Optimal task contribution depends on noise severity.

Abstract

We present a multi-task learning framework to enable the training of one universal incremental dialogue processing model with four tasks of disfluency detection, language modelling, part-of-speech tagging, and utterance segmentation in a simple deep recurrent setting. We show that these tasks provide positive inductive biases to each other with the optimal contribution of each one relying on the severity of the noise from the task. Our live multi-task model outperforms similar individual tasks, delivers competitive performance, and is beneficial for future use in conversational agents in psychiatric treatment.