Enhancing Emotion Recognition in Conversation through Emotional Cross-Modal Fusion and Inter-class Contrastive Learning

TL;DR

This paper introduces a novel cross-modal fusion network with inter-class contrastive learning to improve emotion recognition in conversations, effectively leveraging modality-specific information and reducing redundancy.

Contribution

It proposes a vector connection-based fusion network and a supervised contrastive learning module, enhancing emotion recognition accuracy over previous methods.

Findings

Achieved state-of-the-art results on IEMOCAP and MELD datasets.

Demonstrated the effectiveness of cross-modal fusion with contrastive learning.

Improved focus on modality-specific emotional cues.

Abstract

The purpose of emotion recognition in conversation (ERC) is to identify the emotion category of an utterance based on contextual information. Previous ERC methods relied on simple connections for cross-modal fusion and ignored the information differences between modalities, resulting in the model being unable to focus on modality-specific emotional information. At the same time, the shared information between modalities was not processed to generate emotions. Information redundancy problem. To overcome these limitations, we propose a cross-modal fusion emotion prediction network based on vector connections. The network mainly includes two stages: the multi-modal feature fusion stage based on connection vectors and the emotion classification stage based on fused features. Furthermore, we design a supervised inter-class contrastive learning module based on emotion labels. Experimental results confirm the effectiveness of the proposed method, demonstrating excellent performance on the IEMOCAP and MELD datasets.