Information Theory-Guided Heuristic Progressive Multi-View Coding

TL;DR

This paper introduces a novel information theoretical framework and a progressive multi-view coding method, IPMC, to improve multi-view representation learning by reducing noise, filtering views adaptively, and enhancing discriminative features.

Contribution

It proposes a new information theoretical framework for multi-view learning and develops IPMC, a progressive coding method with three tiers to improve representation quality.

Findings

IPMC outperforms state-of-the-art methods in experiments.

The framework effectively reduces view-specific noise.

The method enhances discriminative representation learning.

Abstract

Multi-view representation learning captures comprehensive information from multiple views of a shared context. Recent works intuitively apply contrastive learning (CL) to learn representations, regarded as a pairwise manner, which is still scalable: view-specific noise is not filtered in learning view-shared representations; the fake negative pairs, where the negative terms are actually within the same class as the positive, and the real negative pairs are coequally treated; and evenly measuring the similarities between terms might interfere with optimization. Importantly, few works research the theoretical framework of generalized self-supervised multi-view learning, especially for more than two views. To this end, we rethink the existing multi-view learning paradigm from the information theoretical perspective and then propose a novel information theoretical framework for generalized multi-view learning. Guided by it, we build a multi-view coding method with a three-tier progressive architecture, namely Information theory-guided heuristic Progressive Multi-view Coding (IPMC). In the distribution-tier, IPMC aligns the distribution between views to reduce view-specific noise. In the set-tier, IPMC builds self-adjusted pools for contrasting, which utilizes a view filter to adaptively modify the pools. Lastly, in the instance-tier, we adopt a designed unified loss to learn discriminative representations and reduce the gradient interference. Theoretically and empirically, we demonstrate the superiority of IPMC over state-of-the-art methods.