Information Theory-Guided Heuristic Progressive Multi-View Coding

TL;DR

This paper introduces a novel information theory-based framework for multi-view learning, addressing limitations of contrastive methods and proposing a hierarchical coding architecture to improve representation quality across multiple views.

Contribution

It develops a theoretical framework for generalized multi-view learning and proposes IPMC, a three-tier progressive coding method guided by information theory.

Findings

IPMC outperforms state-of-the-art methods in experiments.

Theoretical analysis confirms the effectiveness of the hierarchical approach.

Empirical results show improved noise filtering and representation quality.

Abstract

Multi-view representation learning aims to capture comprehensive information from multiple views of a shared context. Recent works intuitively apply contrastive learning to different views in a pairwise manner, which is still scalable: view-specific noise is not filtered in learning view-shared representations; the fake negative pairs, where the negative terms are actually within the same class as the positive, and the real negative pairs are coequally treated; evenly measuring the similarities between terms might interfere with optimization. Importantly, few works study the theoretical framework of generalized self-supervised multi-view learning, especially for more than two views. To this end, we rethink the existing multi-view learning paradigm from the perspective of information theory and then propose a novel information theoretical framework for generalized multi-view learning. Guided by it, we build a multi-view coding method with a three-tier progressive architecture, namely Information theory-guided hierarchical Progressive Multi-view Coding (IPMC). In the distribution-tier, IPMC aligns the distribution between views to reduce view-specific noise. In the set-tier, IPMC constructs self-adjusted contrasting pools, which are adaptively modified by a view filter. Lastly, in the instance-tier, we adopt a designed unified loss to learn representations and reduce the gradient interference. Theoretically and empirically, we demonstrate the superiority of IPMC over state-of-the-art methods.