CloneBot: Personalized Dialogue-Response Predictions

TL;DR

CloneBot introduces a personalized, transformer-based dialogue response prediction model that leverages dense-vector clustering for real-time, long-term context retrieval, enhancing human-like conversational capabilities.

Contribution

The paper presents a novel personalized dialogue-response prediction model using dense-vector clustering and transformer architecture, enabling real-time predictions without additional training.

Findings

Effective use of dense-vector clustering for context retrieval

State-of-the-art transformer-based model on Switchboard corpus

Real-time, training-free utterance prediction

Abstract

Our project task was to create a model that, given a speaker ID, chat history, and an utterance query, can predict the response utterance in a conversation. The model is personalized for each speaker. This task can be a useful tool for building speech bots that talk in a human-like manner in a live conversation. Further, we succeeded at using dense-vector encoding clustering to be able to retrieve relevant historical dialogue context, a useful strategy for overcoming the input limitations of neural-based models when predictions require longer-term references from the dialogue history. In this paper, we have implemented a state-of-the-art model using pre-training and fine-tuning techniques built on transformer architecture and multi-headed attention blocks for the Switchboard corpus. We also show how efficient vector clustering algorithms can be used for real-time utterance predictions that require no training and therefore work on offline and encrypted message histories.