Multi-view Information Bottleneck Without Variational Approximation

TL;DR

This paper introduces a novel multi-view learning method based on the information bottleneck principle that directly optimizes the objective using matrix-based R{\'e}nyi's entropy, avoiding variational approximations.

Contribution

It extends the information bottleneck framework to supervised multi-view learning and employs a matrix-based R{\'e}nyi entropy functional for direct optimization, enhancing robustness.

Findings

Improved robustness to noise and redundant information.

Effective with limited training samples.

Avoids variational approximation or adversarial training.

Abstract

By "intelligently" fusing the complementary information across different views, multi-view learning is able to improve the performance of classification tasks. In this work, we extend the information bottleneck principle to a supervised multi-view learning scenario and use the recently proposed matrix-based R{\'e}nyi's $\alpha$-order entropy functional to optimize the resulting objective directly, without the necessity of variational approximation or adversarial training. Empirical results in both synthetic and real-world datasets suggest that our method enjoys improved robustness to noise and redundant information in each view, especially given limited training samples. Code is available at~\url{https://github.com/archy666/MEIB}.