TVWorld: Foundations for Remote-Control TV Agents

TL;DR

This paper introduces TVWorld, a comprehensive benchmark suite for remote-control TV navigation, revealing limitations of current models and proposing a topology-aware training framework that significantly improves TV navigation performance.

Contribution

The paper presents TVWorld benchmarks, identifies topology awareness as a key challenge, and develops TVTheseus with topology-aware training to advance TV navigation models.

Findings

TVTheseus achieves 68.3% success on TVWorld-N

Topology awareness improves long-horizon TV navigation

TVWorld benchmarks reveal limitations of existing agents

Abstract

Recent large vision-language models (LVLMs) have demonstrated strong potential for device control. However, existing research has primarily focused on point-and-click (PnC) interaction, while remote-control (RC) interaction commonly encountered in everyday TV usage remains largely underexplored. To fill this gap, we introduce \textbf{TVWorld}, an offline graph-based abstraction of real-world TV navigation that enables reproducible and deployment-free evaluation. On this basis, we derive two complementary benchmarks that comprehensively assess TV-use capabilities: \textbf{TVWorld-N} for topology-aware navigation and \textbf{TVWorld-G} for focus-aware grounding. These benchmarks expose a key limitation of existing agents: insufficient topology awareness for focus-based, long-horizon TV navigation. Motivated by this finding, we propose a \emph{Topology-Aware Training} framework that injects topology awareness into LVLMs. Using this framework, we develop \textbf{TVTheseus}, a foundation model specialized for TV navigation. TVTheseus achieves a success rate of $68.3\%$ on TVWorld-N, surpassing strong closed-source baselines such as Gemini 3 Flash and establishing state-of-the-art (SOTA) performance. Additional analyses further provide valuable insights into the development of effective TV-use agents.