Efficient Architectures for High Resolution Vision-Language Models

TL;DR

This paper introduces Pheye, an efficient high-resolution vision-language model architecture that reduces parameters and enhances fine-grained image understanding, especially in scene-text recognition tasks.

Contribution

The paper presents Pheye, a novel architecture that efficiently processes high-resolution images with fewer parameters, improving fine detail recognition in vision-language tasks.

Findings

Pheye achieves high efficiency with fewer parameters.

Pheye performs well in fine-grained image understanding tasks.

Pheye excels in scene-text recognition tasks.

Abstract

Vision-Language Models (VLMs) have recently experienced significant advancements. However, challenges persist in the accurate recognition of fine details within high resolution images, which limits performance in multiple tasks. This work introduces Pheye, a novel architecture that efficiently processes high-resolution images while training fewer parameters than similarly sized VLMs. Notably, Pheye achieves a high efficiency while maintaining strong performance, particularly in tasks that demand fine-grained image understanding and/or the handling of scene-text.