A Survey of Multi-View Representation Learning

TL;DR

This survey comprehensively reviews multi-view representation learning, covering alignment and fusion methods, their theoretical foundations, recent advancements, and key applications in machine learning.

Contribution

It provides an organized overview of existing methods, theories, and applications, highlighting recent developments and categorizing approaches into alignment and fusion strategies.

Findings

Correlation-based alignment methods like CCA are foundational.

Neural network-based multi-view methods have advanced significantly.

Multi-view learning applications span various domains.

Abstract

Recently, multi-view representation learning has become a rapidly growing direction in machine learning and data mining areas. This paper introduces two categories for multi-view representation learning: multi-view representation alignment and multi-view representation fusion. Consequently, we first review the representative methods and theories of multi-view representation learning based on the perspective of alignment, such as correlation-based alignment. Representative examples are canonical correlation analysis (CCA) and its several extensions. Then from the perspective of representation fusion we investigate the advancement of multi-view representation learning that ranges from generative methods including multi-modal topic learning, multi-view sparse coding, and multi-view latent space Markov networks, to neural network-based methods including multi-modal autoencoders, multi-view convolutional neural networks, and multi-modal recurrent neural networks. Further, we also investigate several important applications of multi-view representation learning. Overall, this survey aims to provide an insightful overview of theoretical foundation and state-of-the-art developments in the field of multi-view representation learning and to help researchers find the most appropriate tools for particular applications.