SentiMM: A Multimodal Multi-Agent Framework for Sentiment Analysis in Social Media

TL;DR

SentiMM is a novel multimodal multi-agent framework that enhances social media sentiment analysis by effectively processing heterogeneous data, integrating external knowledge, and achieving superior classification performance.

Contribution

It introduces a multi-agent system for multimodal sentiment analysis and a large-scale dataset with fine-grained sentiment categories.

Findings

SentiMM outperforms existing methods on benchmark datasets.

The framework effectively fuses text and visual data.

Knowledge integration improves sentiment recognition accuracy.

Abstract

With the increasing prevalence of multimodal content on social media, sentiment analysis faces significant challenges in effectively processing heterogeneous data and recognizing multi-label emotions. Existing methods often lack effective cross-modal fusion and external knowledge integration. We propose SentiMM, a novel multi-agent framework designed to systematically address these challenges. SentiMM processes text and visual inputs through specialized agents, fuses multimodal features, enriches context via knowledge retrieval, and aggregates results for final sentiment classification. We also introduce SentiMMD, a large-scale multimodal dataset with seven fine-grained sentiment categories. Extensive experiments demonstrate that SentiMM achieves superior performance compared to state-of-the-art baselines, validating the effectiveness of our structured approach.