Sequential Late Fusion Technique for Multi-modal Sentiment Analysis

TL;DR

This paper introduces a novel multi-head attention LSTM-based fusion technique for multi-modal sentiment analysis, leveraging text, audio, and visual data to improve emotion recognition accuracy.

Contribution

The work presents a new fusion method using multi-head attention LSTM networks specifically designed for multi-modal sentiment analysis.

Findings

Improved sentiment classification accuracy on MOSI dataset

Effective integration of text, audio, and visual modalities

Demonstrated superiority over existing fusion techniques

Abstract

Multi-modal sentiment analysis plays an important role for providing better interactive experiences to users. Each modality in multi-modal data can provide different viewpoints or reveal unique aspects of a user's emotional state. In this work, we use text, audio and visual modalities from MOSI dataset and we propose a novel fusion technique using a multi-head attention LSTM network. Finally, we perform a classification task and evaluate its performance.