# Fusion to Enhance: Fusion Visual Encoder to Enhance Multimodal Language Model

## TL;DR

This paper introduces FtZ, a new vision framework combining multiple encoders with cross-attention to improve fine-grained visual understanding in multimodal language models, outperforming existing methods.

## Contribution

The paper proposes a novel multi-encoder fusion framework, FtZ, that enhances visual perception in multimodal models by combining semantic and perceptual encoders via cross-attention.

## Key findings

- Significant improvements on TextVQA, POPE, MMMU, MME, MM-Vet benchmarks.
- Outperforms single-encoder and existing fusion methods.
- Validates the effectiveness of heterogeneous encoder composition.

## Abstract

Multimodal Large Language Models (MLLMs) have made significant progress in bridging visual perception with high-level textual reasoning. However, they face a fundamental contradiction: while excelling at complex semantic understanding, these models often fail at basic visual tasks that require precise detail perception. This deficiency primarily stems from the prevalent architectural reliance on a single vision encoder optimized for high-level semantic alignment, which inherently sacrifices the ability to capture fine-grained visual information. To address this issue, we introduce Fusion to Enhance (FtZ), a novel vision tower framework. FtZ moves beyond the single-encoder design by innovatively composing a semantically powerful anchor encoder with a perception-rich augmenting encoder via a lightweight Multi-Head Cross-Attention mechanism. Experimental results demonstrate that on several challenging benchmarks demanding fine-grained visual understanding, such as TextVQA, POPE, MMMU, MME and MM-Vet, our FtZ model significantly outperforms baselines that use only a single encoder or existing feature fusion methods. This work proves that composing heterogeneous expert encoders is an efficient and effective path to overcoming the visual perception bottleneck in current MLLMs, offering a new design paradigm for building next-generation AI systems with stronger perceptual capabilities.