Data Uncertainty-Aware Learning for Multimodal Aspect-based Sentiment Analysis

TL;DR

This paper introduces UA-MABSA, a novel approach for multimodal aspect-based sentiment analysis that accounts for data uncertainty by weighting samples based on quality, leading to improved performance especially on noisy data.

Contribution

The paper proposes a data uncertainty-aware method that assesses and incorporates data quality into the learning process for MABSA, which was not addressed in prior work.

Findings

Achieves state-of-the-art results on Twitter-2015 dataset.

Effective quality assessment improves model focus on high-quality samples.

Weighted loss based on data quality enhances sentiment analysis accuracy.

Abstract

As a fine-grained task, multimodal aspect-based sentiment analysis (MABSA) mainly focuses on identifying aspect-level sentiment information in the text-image pair. However, we observe that it is difficult to recognize the sentiment of aspects in low-quality samples, such as those with low-resolution images that tend to contain noise. And in the real world, the quality of data usually varies for different samples, such noise is called data uncertainty. But previous works for the MABSA task treat different quality samples with the same importance and ignored the influence of data uncertainty. In this paper, we propose a novel data uncertainty-aware multimodal aspect-based sentiment analysis approach, UA-MABSA, which weighted the loss of different samples by the data quality and difficulty. UA-MABSA adopts a novel quality assessment strategy that takes into account both the image quality and the aspect-based cross-modal relevance, thus enabling the model to pay more attention to high-quality and challenging samples. Extensive experiments show that our method achieves state-of-the-art (SOTA) performance on the Twitter-2015 dataset. Further analysis demonstrates the effectiveness of the quality assessment strategy.