PhaseWin Search Framework Enable Efficient Object-Level Interpretation

TL;DR

PhaseWin introduces a fast, scalable search algorithm for object-level attribution that maintains high faithfulness while significantly reducing computational costs, enabling practical deployment in real-world applications.

Contribution

It proposes a novel phase-window search algorithm that approximates greedy attribution with near-linear complexity, improving efficiency without sacrificing accuracy.

Findings

Achieves over 95% of greedy attribution faithfulness with only 20% of the computational budget.

Outperforms existing attribution methods across object detection and visual grounding tasks.

Establishes a new state of the art in scalable, high-faithfulness attribution for object-level models.

Abstract

Attribution is essential for interpreting object-level foundation models. Recent methods based on submodular subset selection have achieved high faithfulness, but their efficiency limitations hinder practical deployment in real-world scenarios. To address this, we propose PhaseWin, a novel phase-window search algorithm that enables faithful region attribution with near-linear complexity. PhaseWin replaces traditional quadratic-cost greedy selection with a phased coarse-to-fine search, combining adaptive pruning, windowed fine-grained selection, and dynamic supervision mechanisms to closely approximate greedy behavior while dramatically reducing model evaluations. Theoretically, PhaseWin retains near-greedy approximation guarantees under mild monotone submodular assumptions. Empirically, PhaseWin achieves over 95% of greedy attribution faithfulness using only 20% of the computational budget, and consistently outperforms other attribution baselines across object detection and visual grounding tasks with Grounding DINO and Florence-2. PhaseWin establishes a new state of the art in scalable, high-faithfulness attribution for object-level multimodal models.