Don't Shoot The Breeze: Topic Continuity Model Using Nonlinear Naive Bayes With Attention

TL;DR

This paper introduces a nonlinear Naive Bayes-based topic continuity model with attention, improving LLM chatbot responses by maintaining conversation coherence over any length with interpretability and efficiency.

Contribution

The paper presents a novel nonlinear Naive Bayes model with attention for assessing topic continuity, capable of handling conversations of arbitrary length with improved accuracy.

Findings

Outperforms traditional methods in maintaining topic continuity.

Handles conversations of any length with linear time complexity.

Enhances interpretability of topic assessment.

Abstract

Utilizing Large Language Models (LLM) as chatbots in diverse business scenarios often presents the challenge of maintaining topic continuity. Abrupt shifts in topics can lead to poor user experiences and inefficient utilization of computational resources. In this paper, we present a topic continuity model aimed at assessing whether a response aligns with the initial conversation topic. Our model is built upon the expansion of the corresponding natural language understanding (NLU) model into quantifiable terms using a Naive Bayes approach. Subsequently, we have introduced an attention mechanism and logarithmic nonlinearity to enhance its capability to capture topic continuity. This approach allows us to convert the NLU model into an interpretable analytical formula. In contrast to many NLU models constrained by token limits, our proposed model can seamlessly handle conversations of any length with linear time complexity. Furthermore, the attention mechanism significantly improves the model's ability to identify topic continuity in complex conversations. According to our experiments, our model consistently outperforms traditional methods, particularly in handling lengthy and intricate conversations. This unique capability offers us an opportunity to ensure the responsible and interpretable use of LLMs.