Dynamic Graph Neural ODE Network for Multi-modal Emotion Recognition in Conversation

TL;DR

This paper introduces DGODE, a novel dynamic graph neural ODE network that captures temporal dependencies and reduces overfitting in multimodal emotion recognition, leading to improved performance and deeper GCNs.

Contribution

The paper proposes DGODE, integrating adaptive mixhop and graph ODE to model continuous emotion dynamics and mitigate overfitting in multimodal MERC.

Findings

DGODE outperforms baseline models on two datasets.

It effectively alleviates overfitting and over-smoothing.

Enables construction of deeper GCN networks.

Abstract

Multimodal emotion recognition in conversation (MERC) refers to identifying and classifying human emotional states by combining data from multiple different modalities (e.g., audio, images, text, video, etc.). Most existing multimodal emotion recognition methods use GCN to improve performance, but existing GCN methods are prone to overfitting and cannot capture the temporal dependency of the speaker's emotions. To address the above problems, we propose a Dynamic Graph Neural Ordinary Differential Equation Network (DGODE) for MERC, which combines the dynamic changes of emotions to capture the temporal dependency of speakers' emotions, and effectively alleviates the overfitting problem of GCNs. Technically, the key idea of DGODE is to utilize an adaptive mixhop mechanism to improve the generalization ability of GCNs and use the graph ODE evolution network to characterize the continuous dynamics of node representations over time and capture temporal dependencies. Extensive experiments on two publicly available multimodal emotion recognition datasets demonstrate that the proposed DGODE model has superior performance compared to various baselines. Furthermore, the proposed DGODE can also alleviate the over-smoothing problem, thereby enabling the construction of a deep GCN network.