Affective Multimodal Agents with Proactive Knowledge Grounding for Emotionally Aligned Marketing Dialogue

TL;DR

AffectMind is a multimodal dialogue agent designed for emotionally aligned marketing conversations, using proactive knowledge grounding and emotion-aware strategies to improve engagement and persuasion.

Contribution

The paper introduces AffectMind, a novel multimodal affective dialogue system with proactive reasoning and dynamic knowledge grounding for emotionally aligned marketing interactions.

Findings

Outperforms LLM baselines in emotional consistency (+26%)

Increases persuasive success rate (+19%)

Enhances long-term user engagement (+23%)

Abstract

Recent advances in large language models (LLMs) have enabled fluent dialogue systems, but most remain reactive and struggle in emotionally rich, goal-oriented settings such as marketing conversations. To address this limitation, we propose AffectMind, a multimodal affective dialogue agent that performs proactive reasoning and dynamic knowledge grounding to sustain emotionally aligned and persuasive interactions. AffectMind combines three components: a Proactive Knowledge Grounding Network (PKGN) that continuously updates factual and affective context from text, vision, and prosody; an Emotion--Intent Alignment Model (EIAM) that jointly models user emotion and purchase intent to adapt persuasion strategies; and a Reinforced Discourse Loop (RDL) that optimizes emotional coherence and engagement via reinforcement signals from user responses. Experiments on two newly curated marketing dialogue datasets, MM-ConvMarket and AffectPromo, show that AffectMind outperforms strong LLM-based baselines in emotional consistency (+26\%), persuasive success rate (+19\%), and long-term user engagement (+23\%), highlighting emotion-grounded proactivity as a key capability for commercial multimodal agents.