Multi-channel Attentive Graph Convolutional Network With Sentiment Fusion For Multimodal Sentiment Analysis

TL;DR

This paper introduces MAGCN, a novel multimodal sentiment analysis model that integrates sentimental knowledge into inter-modality learning using graph convolution and self-attention mechanisms, achieving competitive results.

Contribution

The paper proposes a multi-channel attentive graph convolutional network that effectively fuses sentimental knowledge with multimodal features for improved sentiment analysis.

Findings

Achieves superior accuracy and F1 scores on three datasets.

Effectively models inter-modality dynamics with graph convolution.

Incorporates sentimental knowledge to enhance feature representations.

Abstract

Nowadays, with the explosive growth of multimodal reviews on social media platforms, multimodal sentiment analysis has recently gained popularity because of its high relevance to these social media posts. Although most previous studies design various fusion frameworks for learning an interactive representation of multiple modalities, they fail to incorporate sentimental knowledge into inter-modality learning. This paper proposes a Multi-channel Attentive Graph Convolutional Network (MAGCN), consisting of two main components: cross-modality interactive learning and sentimental feature fusion. For cross-modality interactive learning, we exploit the self-attention mechanism combined with densely connected graph convolutional networks to learn inter-modality dynamics. For sentimental feature fusion, we utilize multi-head self-attention to merge sentimental knowledge into inter-modality feature representations. Extensive experiments are conducted on three widely-used datasets. The experimental results demonstrate that the proposed model achieves competitive performance on accuracy and F1 scores compared to several state-of-the-art approaches.