Toward Natural and Companionable Virtual Agents via Cross-Temporal Emotional Modeling

TL;DR

This paper introduces Cross-Temporal Emotion Modeling (CTEM), a framework for virtual agents that enhances long-term, natural, and emotionally coherent interactions by linking past behaviors to current emotional states.

Contribution

The paper proposes a novel CTEM framework that integrates long-term behavioral history with real-time emotional expression, improving virtual agent companionship.

Findings

Auri, the instantiated agent, showed improved naturalness, coherence, and emotional harmony in a 21-day study.

CTEM enables dynamic updating of emotional states based on past interactions and user feedback.

Abstract

Recent advances in foundation models have enabled conversational agents that aim for sustained companionship rather than mere task completion. Yet most still remain unable to support natural, long-term companion-like interactions, resulting in experiences that feel episodic and inauthentic. We argue that current agents overlooked cross-temporal modeling of agents' social behaviors and internal emotions: generated behaviors rarely influence an agent's emotional state, and emotional states seldom shape subsequent behaviors. We present Cross-Temporal Emotion Modeling (CTEM), a framework that links long-term behavioral history to moment-to-moment emotional expression. CTEM establishes a closed loop where past experiences update an evolving emotional state; this state conditions immediate interactions; and user feedback continually revises both memory and emotional state, enabling reflection and anticipation. We instantiate CTEM as Auri, a companion agent on an instant-messaging platform, and report a 21-day in-the-wild study showing that CTEM shows improvements in perceived naturalness, coherence, and emotional harmony.