Empathic Prompting: Non-Verbal Context Integration for Multimodal LLM Conversations

TL;DR

Empathic Prompting introduces a modular framework that enhances multimodal LLM conversations by unobtrusively integrating users' non-verbal emotional cues, improving conversational fluidity without explicit user control.

Contribution

It is the first to embed implicit non-verbal emotional context into LLM prompts using facial expression recognition, enhancing multimodal human-AI interaction.

Findings

Consistent integration of non-verbal cues into LLM outputs.

Participants reported improved conversational fluidity.

Framework is scalable and adaptable to additional non-verbal modules.

Abstract

We present Empathic Prompting, a novel framework for multimodal human-AI interaction that enriches Large Language Model (LLM) conversations with implicit non-verbal context. The system integrates a commercial facial expression recognition service to capture users' emotional cues and embeds them as contextual signals during prompting. Unlike traditional multimodal interfaces, empathic prompting requires no explicit user control; instead, it unobtrusively augments textual input with affective information for conversational and smoothness alignment. The architecture is modular and scalable, allowing integration of additional non-verbal modules. We describe the system design, implemented through a locally deployed DeepSeek instance, and report a preliminary service and usability evaluation (N=5). Results show consistent integration of non-verbal input into coherent LLM outputs, with participants highlighting conversational fluidity. Beyond this proof of concept, empathic prompting points to applications in chatbot-mediated communication, particularly in domains like healthcare or education, where users' emotional signals are critical yet often opaque in verbal exchanges.