Image Tiling for High-Resolution Reasoning: Balancing Local Detail with Global Context

TL;DR

This paper reproduces and analyzes the Monkey Vision-Language Model's image tiling approach for high-resolution image understanding, confirming its effectiveness and exploring the impact of global context inclusion on performance.

Contribution

It provides a detailed reproduction of the Monkey VLM's tiling method and extends the analysis by investigating the role of global context in high-resolution multimodal models.

Findings

Tiling recovers local visual details effectively.

Including global context influences model performance.

Results vary depending on task and tile size.

Abstract

Reproducibility remains a cornerstone of scientific progress, yet complex multimodal models often lack transparent implementation details and accessible training infrastructure. In this work, we present a detailed reproduction and critical analysis of the Monkey Vision-Language Model (VLM) (Li et al. 2023b) published in CVPR24, a recent approach to high-resolution image understanding via image tiling. The original paper proposed splitting large images into tiles to recover fine-grained visual details while maintaining computational efficiency. Our study replicates this strategy using open checkpoints and reimplements the training pipeline. We confirm the key finding of the original Monkey VLM work, namely that tiling effectively recovers local details. We then extend this work further, by investigating the effect of the inclusion of the global context, which provide practical insights for future high-resolution multimodal modeling. However, we also report deviations in the results, with the magnitude of these effects depending heavily on task type and tile granularity.