EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Models

TL;DR

EmoBench-M is a comprehensive benchmark designed to evaluate multimodal large language models' emotional intelligence across diverse scenarios, revealing significant performance gaps and guiding future improvements.

Contribution

The paper introduces EmoBench-M, a new psychological theory-based benchmark for assessing MLLMs' emotional intelligence in multimodal, dynamic contexts.

Findings

Top models like Gemini-3.0-Pro and GPT-5.2 score 70.5 and 66.5 respectively on EmoBench-M.

Specialized models such as AffectGPT show uneven performance across different emotional scenarios.

There is a substantial gap between current MLLMs and human-level emotional intelligence.

Abstract

With the integration of multimodal large language models (MLLMs) into robotic systems and AI applications, embedding emotional intelligence (EI) capabilities is essential for enabling these models to perceive, interpret, and respond to human emotions effectively in real-world scenarios. Existing static, text-based, or text-image benchmarks overlook the multimodal complexities of real interactions and fail to capture the dynamic, context-dependent nature of emotional expressions, rendering them inadequate for evaluating MLLMs' EI capabilities. To address these limitations, we introduce EmoBench-M, a systematic benchmark grounded in established psychological theories, designed to evaluate MLLMs across 13 evaluation scenarios spanning three hierarchical dimensions: foundational emotion recognition (FER), conversational emotion understanding (CEU), and socially complex emotion analysis (SCEA). Evaluation was conducted on 27 state-of-the-art MLLMs, using both objective task-specific metrics and LLM-based evaluation, revealing a substantial performance gap relative to human-level competence. Even the best performing models, Gemini-3.0-Pro and GPT-5.2, achieve the highest scores on EmoBench-M, 70.5 and 66.5 points respectively. Specialized models such as AffectGPT exhibit uneven performance across EmoBench-M, demonstrating strengths in certain scenarios but generally lacking comprehensive emotional intelligence. By providing a comprehensive, multimodal evaluation framework, EmoBench-M captures both the strengths and weaknesses of current MLLMs across diverse emotional contexts. All benchmark resources, including datasets and code, are publicly available at https://emo-gml.github.io/, facilitating further research and advancement in MLLM emotional intelligence.