Multimodal Multi-turn Conversation Stance Detection: A Challenge Dataset and Effective Model

TL;DR

This paper introduces a new multimodal multi-turn conversational stance detection dataset and a novel large language model framework that jointly analyzes text and images to identify public opinion in social media conversations.

Contribution

The paper presents the first dataset capturing multi-party conversational contexts for multimodal stance detection and proposes an effective large language model approach for joint multimodal stance analysis.

Findings

State-of-the-art performance on MmMtCSD dataset

Effective joint modeling of text and images for stance detection

Addresses multi-turn conversational scenarios in social media

Abstract

Stance detection, which aims to identify public opinion towards specific targets using social media data, is an important yet challenging task. With the proliferation of diverse multimodal social media content including text, and images multimodal stance detection (MSD) has become a crucial research area. However, existing MSD studies have focused on modeling stance within individual text-image pairs, overlooking the multi-party conversational contexts that naturally occur on social media. This limitation stems from a lack of datasets that authentically capture such conversational scenarios, hindering progress in conversational MSD. To address this, we introduce a new multimodal multi-turn conversational stance detection dataset (called MmMtCSD). To derive stances from this challenging dataset, we propose a novel multimodal large language model stance detection framework (MLLM-SD), that learns joint stance representations from textual and visual modalities. Experiments on MmMtCSD show state-of-the-art performance of our proposed MLLM-SD approach for multimodal stance detection. We believe that MmMtCSD will contribute to advancing real-world applications of stance detection research.