A Survey on Multi-modal Summarization

TL;DR

This survey reviews the current state of multi-modal summarization, covering various modalities, evaluation metrics, datasets, challenges, and future research directions in the field.

Contribution

It provides a comprehensive overview of existing MMS research, highlighting key challenges and proposing future directions for the field.

Findings

Summarization techniques vary across modalities.

Evaluation metrics and datasets are diverse and evolving.

Challenges include modality integration and data quality.

Abstract

The new era of technology has brought us to the point where it is convenient for people to share their opinions over an abundance of platforms. These platforms have a provision for the users to express themselves in multiple forms of representations, including text, images, videos, and audio. This, however, makes it difficult for users to obtain all the key information about a topic, making the task of automatic multi-modal summarization (MMS) essential. In this paper, we present a comprehensive survey of the existing research in the area of MMS, covering various modalities like text, image, audio, and video. Apart from highlighting the different evaluation metrics and datasets used for the MMS task, our work also discusses the current challenges and future directions in this field.