EmoVerse: A MLLMs-Driven Emotion Representation Dataset for Interpretable Visual Emotion Analysis

TL;DR

EmoVerse is a large-scale, open-source dataset with multi-layered, interpretable annotations for visual emotion analysis, enabling detailed emotion reasoning grounded in visual regions.

Contribution

This work introduces EmoVerse, a novel dataset with knowledge-graph-inspired annotations, dual emotion representations, and an interpretable model for explainable visual emotion analysis.

Findings

Over 219,000 images annotated with B-A-S triplets and dual emotion labels.

A multi-stage pipeline improves annotation reliability with minimal human effort.

An interpretable model effectively maps visual cues to emotion representations and explanations.

Abstract

Visual Emotion Analysis (VEA) aims to bridge the affective gap between visual content and human emotional responses. Despite its promise, progress in this field remains limited by the lack of open-source and interpretable datasets. Most existing studies assign a single discrete emotion label to an entire image, offering limited insight into how visual elements contribute to emotion. In this work, we introduce EmoVerse, a large-scale open-source dataset that enables interpretable visual emotion analysis through multi-layered, knowledge-graph-inspired annotations. By decomposing emotions into Background-Attribute-Subject (B-A-S) triplets and grounding each element to visual regions, EmoVerse provides word-level and subject-level emotional reasoning. With over 219k images, the dataset further includes dual annotations in Categorical Emotion States (CES) and Dimensional Emotion Space (DES), facilitating unified discrete and continuous emotion representation. A novel multi-stage pipeline ensures high annotation reliability with minimal human effort. Finally, we introduce an interpretable model that maps visual cues into DES representations and provides detailed attribution explanations. Together, the dataset, pipeline, and model form a comprehensive foundation for advancing explainable high-level emotion understanding.