Dialogue Response Prefetching Based on Semantic Similarity and Prediction Confidence of Language Model

TL;DR

This paper introduces a prediction confidence model for dialogue systems that estimates semantic similarity to determine the feasibility of prefetching responses, aiming to reduce user-perceived latency.

Contribution

The study proposes a novel PCM that assesses semantic similarity and prediction confidence to improve response prefetching in spoken dialogue systems.

Findings

PCM effectively estimates the likelihood of successful prefetching.

Semantic similarity correlates with prediction accuracy.

The approach reduces user-perceived latency in dialogue systems.

Abstract

Prefetching of dialogue responses has been investigated to reduce user-perceived latency (UPL), which refers to the user's waiting time before receiving the system's response, in spoken dialogue systems. To reduce the UPL, it is necessary to predict complete user utterances before the end of the user's speech, typically by language models, to prepare prefetched dialogue responses. In this study, we proposed a prediction confidence model (PCM) that determines whether prefetching is possible or not by estimating the semantic similarity between the predicted complete user utterance and the complete user utterance. We evaluated our PCM based on the differences between the predicted complete user utterance and the complete user utterance.