Learning to generate and corr- uh I mean repair language in real-time

TL;DR

This paper presents a probabilistic model for incremental language generation and repair in conversation, demonstrating high accuracy in matching gold standards and generating natural self-repairs, advancing conversational AI capabilities.

Contribution

It introduces a grammar-based probabilistic model capable of incremental generation and self-repair, trained on a semantic goal, with strong evaluation results.

Findings

78% exact match with gold candidates

85% correct self-repair generation

High naturalness and grammaticality in human evaluation

Abstract

In conversation, speakers produce language incrementally, word by word, while continuously monitoring the appropriateness of their own contribution in the dynamically unfolding context of the conversation; and this often leads them to repair their own utterance on the fly. This real-time language processing capacity is furthermore crucial to the development of fluent and natural conversational AI. In this paper, we use a previously learned Dynamic Syntax grammar and the CHILDES corpus to develop, train and evaluate a probabilistic model for incremental generation where input to the model is a purely semantic generation goal concept in Type Theory with Records (TTR). We show that the model's output exactly matches the gold candidate in 78% of cases with a ROUGE-l score of 0.86. We further do a zero-shot evaluation of the ability of the same model to generate self-repairs when the generation goal changes mid-utterance. Automatic evaluation shows that the model can generate self-repairs correctly in 85% of cases. A small human evaluation confirms the naturalness and grammaticality of the generated self-repairs. Overall, these results further highlight the generalisation power of grammar-based models and lay the foundations for more controllable, and naturally interactive conversational AI systems.