Proactive Conversational Agents with Inner Thoughts

TL;DR

This paper introduces the Inner Thoughts framework for proactive conversational AI, enabling it to generate continuous internal thoughts to engage more naturally in multi-party conversations, surpassing previous reactive methods.

Contribution

It proposes a novel Inner Thoughts framework inspired by linguistics and psychology, allowing AI to proactively contribute by modeling its intrinsic motivations.

Findings

Framework improves anthropomorphism and coherence

User studies show increased perceived intelligence

Significantly outperforms existing baselines

Abstract

One of the long-standing aspirations in conversational AI is to allow them to autonomously take initiatives in conversations, i.e., being proactive. This is especially challenging for multi-party conversations. Prior NLP research focused mainly on predicting the next speaker from contexts like preceding conversations. In this paper, we demonstrate the limitations of such methods and rethink what it means for AI to be proactive in multi-party, human-AI conversations. We propose that just like humans, rather than merely reacting to turn-taking cues, a proactive AI formulates its own inner thoughts during a conversation, and seeks the right moment to contribute. Through a formative study with 24 participants and inspiration from linguistics and cognitive psychology, we introduce the Inner Thoughts framework. Our framework equips AI with a continuous, covert train of thoughts in parallel to the overt communication process, which enables it to proactively engage by modeling its intrinsic motivation to express these thoughts. We instantiated this framework into two real-time systems: an AI playground web app and a chatbot. Through a technical evaluation and user studies with human participants, our framework significantly surpasses existing baselines on aspects like anthropomorphism, coherence, intelligence, and turn-taking appropriateness.