A survey on Self Supervised learning approaches for improving Multimodal representation learning

TL;DR

This survey reviews various self-supervised learning methods for multimodal representation learning, highlighting approaches like cross-modal generation, pretraining, cyclic translation, and unimodal label generation to enhance multimodal models.

Contribution

It provides a comprehensive overview of the state-of-the-art self-supervised techniques specifically tailored for multimodal learning, aggregating diverse methods from recent literature.

Findings

Identifies key self-supervised approaches for multimodal learning

Highlights the effectiveness of cross-modal pretraining and generation

Provides a structured taxonomy of methods

Abstract

Recently self supervised learning has seen explosive growth and use in variety of machine learning tasks because of its ability to avoid the cost of annotating large-scale datasets. This paper gives an overview for best self supervised learning approaches for multimodal learning. The presented approaches have been aggregated by extensive study of the literature and tackle the application of self supervised learning in different ways. The approaches discussed are cross modal generation, cross modal pretraining, cyclic translation, and generating unimodal labels in self supervised fashion.