How Vision-Language Tasks Benefit from Large Pre-trained Models: A Survey

TL;DR

This survey reviews how large pre-trained models have significantly advanced vision-language tasks like captioning and question answering, highlighting recent progress, challenges, and future research directions.

Contribution

It provides a comprehensive overview of the impact of pre-trained models on vision-language tasks, including challenges, recent advances, and potential risks.

Findings

Pre-trained models improve performance in vision-language tasks.

Challenges remain despite advancements in pre-trained models.

Future research should address inherent limitations of pre-trained models.

Abstract

The exploration of various vision-language tasks, such as visual captioning, visual question answering, and visual commonsense reasoning, is an important area in artificial intelligence and continuously attracts the research community's attention. Despite the improvements in overall performance, classic challenges still exist in vision-language tasks and hinder the development of this area. In recent years, the rise of pre-trained models is driving the research on vision-language tasks. Thanks to the massive scale of training data and model parameters, pre-trained models have exhibited excellent performance in numerous downstream tasks. Inspired by the powerful capabilities of pre-trained models, new paradigms have emerged to solve the classic challenges. Such methods have become mainstream in current research with increasing attention and rapid advances. In this paper, we present a comprehensive overview of how vision-language tasks benefit from pre-trained models. First, we review several main challenges in vision-language tasks and discuss the limitations of previous solutions before the era of pre-training. Next, we summarize the recent advances in incorporating pre-trained models to address the challenges in vision-language tasks. Finally, we analyze the potential risks associated with the inherent limitations of pre-trained models and discuss possible solutions, attempting to provide future research directions.