Tri-Subspaces Disentanglement for Multimodal Sentiment Analysis

TL;DR

This paper introduces a novel Tri-Subspace Disentanglement framework for multimodal sentiment analysis, explicitly modeling shared and private features across language, visual, and acoustic modalities to improve sentiment inference accuracy.

Contribution

The paper proposes a new TSD framework with subspace factorization, decoupling supervision, and a subspace-aware attention module, advancing multimodal sentiment analysis by better capturing cross-modal signals.

Findings

Achieves state-of-the-art results on CMU-MOSI and CMU-MOSEI datasets.

Demonstrates effective disentanglement of shared and private modality features.

Enhances cross-modal sentiment modeling through structured subspace fusion.

Abstract

Multimodal Sentiment Analysis (MSA) integrates language, visual, and acoustic modalities to infer human sentiment. Most existing methods either focus on globally shared representations or modality-specific features, while overlooking signals that are shared only by certain modality pairs. This limits the expressiveness and discriminative power of multimodal representations. To address this limitation, we propose a Tri-Subspace Disentanglement (TSD) framework that explicitly factorizes features into three complementary subspaces: a common subspace capturing global consistency, submodally-shared subspaces modeling pairwise cross-modal synergies, and private subspaces preserving modality-specific cues. To keep these subspaces pure and independent, we introduce a decoupling supervisor together with structured regularization losses. We further design a Subspace-Aware Cross-Attention (SACA) fusion module that adaptively models and integrates information from the three subspaces to obtain richer and more robust representations. Experiments on CMU-MOSI and CMU-MOSEI demonstrate that TSD achieves state-of-the-art performance across all key metrics, reaching 0.691 MAE on CMU-MOSI and 54.9% ACC-7 on CMU-MOSEI, and also transfers well to multimodal intent recognition tasks. Ablation studies confirm that tri-subspace disentanglement and SACA jointly enhance the modeling of multi-granular cross-modal sentiment cues.