On the Multi-View Information Bottleneck Representation

TL;DR

This paper extends the information bottleneck framework to multi-view learning, proposing two novel methods for different overlap scenarios, improving performance and scalability in multi-view classification tasks.

Contribution

It introduces two new formulations for multi-view information bottleneck, addressing complexity issues and demonstrating improved empirical performance over existing methods.

Findings

Proposed methods outperform state-of-the-art in multi-view classification.

Extended ADMM solver with proven convergence and scalability.

Effective in scenarios with both high and low view overlap.

Abstract

In this work, we generalize the information bottleneck (IB) approach to the multi-view learning context. The exponentially growing complexity of the optimal representation motivates the development of two novel formulations with more favorable performance-complexity tradeoffs. The first approach is based on forming a stochastic consensus and is suited for scenarios with significant {\em representation overlap} between the different views. The second method, relying on incremental updates, is tailored for the other extreme scenario with minimal representation overlap. In both cases, we extend our earlier work on the alternating directional methods of multiplier (ADMM) solver and establish its convergence and scalability. Empirically, we find that the proposed methods outperform state-of-the-art approaches in multi-view classification problems under a broad range of modelling parameters.