FANoise: Singular Value-Adaptive Noise Modulation for Robust Multimodal Representation Learning

TL;DR

FANoise introduces a dynamic, feature-adaptive noise injection method for multimodal representation learning, improving robustness and performance by aligning noise modulation with training dynamics.

Contribution

The paper proposes FANoise, a novel noise injection strategy that adapts to feature distributions during training, enhancing multimodal learning robustness and effectiveness.

Findings

FANoise consistently improves performance across multiple multimodal tasks.

The method effectively balances noise benefits and drawbacks during training.

Experimental results show enhanced robustness of learned representations.

Abstract

Representation learning is fundamental to modern machine learning, powering applications such as text retrieval and multimodal understanding. However, learning robust and generalizable representations remains challenging. While prior work has demonstrated that active noise injection, a form of data augmentation, can enhance encoding performance, most existing methods rely on heuristic or static noise, overlooking the dynamic nature of feature distributions during training. In this work, we systematically study the role of noise in representation learning from both gradient-based and feature distribution perspectives, using InfoNCE loss as a representative example. Focusing on multimodal representation learning, we propose FANoise, a novel feature-adaptive noise injection strategy. By leveraging the dynamics of contrastive learning, FANoise effectively mitigates the negative impacts of noise while preserving its benefits. Under this theoretically grounded framework, comprehensive experiments demonstrate that FANoise consistently improves overall performance on multimodal tasks across various base VLM models.