LLM-Driven Multimodal Opinion Expression Identification

TL;DR

This paper introduces a multimodal opinion expression identification task leveraging large language models, combining speech and text to improve detection accuracy and achieve state-of-the-art results.

Contribution

It presents a novel multimodal OEI task, new datasets, and an LLM-driven method that significantly enhances opinion expression detection performance.

Findings

MOEI improves opinion detection accuracy

Proposed method outperforms existing approaches by 9.20%

Achieves state-of-the-art results on benchmark datasets

Abstract

Opinion Expression Identification (OEI) is essential in NLP for applications ranging from voice assistants to depression diagnosis. This study extends OEI to encompass multimodal inputs, underlining the significance of auditory cues in delivering emotional subtleties beyond the capabilities of text. We introduce a novel multimodal OEI (MOEI) task, integrating text and speech to mirror real-world scenarios. Utilizing CMU MOSEI and IEMOCAP datasets, we construct the CI-MOEI dataset. Additionally, Text-to-Speech (TTS) technology is applied to the MPQA dataset to obtain the CIM-OEI dataset. We design a template for the OEI task to take full advantage of the generative power of large language models (LLMs). Advancing further, we propose an LLM-driven method STOEI, which combines speech and text modal to identify opinion expressions. Our experiments demonstrate that MOEI significantly improves the performance while our method outperforms existing methods by 9.20\% and obtains SOTA results.