Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Review

TL;DR

This paper provides a comprehensive survey of multimodal large language models for text-rich image understanding, covering their architectures, performance benchmarks, and future research directions.

Contribution

It offers a systematic review of nearly all TIU MLLMs, detailing their development, performance, and highlighting challenges and future opportunities.

Findings

Summarizes the evolution and architecture of TIU MLLMs.

Reviews model performance on mainstream benchmarks.

Discusses challenges and future research directions.

Abstract

The recent emergence of Multi-modal Large Language Models (MLLMs) has introduced a new dimension to the Text-rich Image Understanding (TIU) field, with models demonstrating impressive and inspiring performance. However, their rapid evolution and widespread adoption have made it increasingly challenging to keep up with the latest advancements. To address this, we present a systematic and comprehensive survey to facilitate further research on TIU MLLMs. Initially, we outline the timeline, architecture, and pipeline of nearly all TIU MLLMs. Then, we review the performance of selected models on mainstream benchmarks. Finally, we explore promising directions, challenges, and limitations within the field.