Video-Browser: Towards Agentic Open-web Video Browsing

TL;DR

This paper introduces Video-Browser, a new agentic framework for open-web video browsing that balances visual perception and efficiency, significantly improving performance and reducing costs in open-ended video exploration.

Contribution

We formalize the task of Agentic Video Browsing, propose the Video-Browser framework with Pyramidal Perception, and establish a benchmark for open-ended video exploration.

Findings

Achieved 37.5% relative improvement over baseline methods.

Reduced token consumption by 58.3% compared to direct visual inference.

Established a foundation for verifiable open-web video research.

Abstract

The evolution of autonomous agents is redefining information seeking, transitioning from passive retrieval to proactive, open-ended web research. However, a significant modality gap remains in processing the web's most dynamic and information-dense modality: video. In this paper, we first formalize the task of Agentic Video Browsing and introduce Video-BrowseComp, a benchmark evaluating open-ended agentic browsing tasks that enforce a mandatory dependency on videos. We observe that current paradigms struggle to reconcile the scale of open-ended video exploration with the need for fine-grained visual verification. Direct visual inference (e.g., RAG) maximizes perception but incurs prohibitive context costs, while text-centric summarization optimizes efficiency but often misses critical visual details required for accurate grounding. To address this, we propose Video-Browser, a novel agent leveraging Pyramidal Perception, filtering with cheap metadata and zooming in with expensive visual perception only when necessary. Experiments demonstrate that our approach achieves a 37.5% relative improvement while reducing token consumption by 58.3% compared to Direct visual inference, establishing a foundation for verifiable open-web video research. We open-source all codes, benchmark at {https://anonymous.4open.science/r/VideoBrowser} and {https://github.com/chrisx599/Video-Browser}.