CostNav: A Navigation Benchmark for Real-World Economic-Cost Evaluation of Physical AI Agents

TL;DR

CostNav introduces a comprehensive economic navigation benchmark that evaluates physical AI agents based on real-world business costs and revenues, highlighting the gap between research success and commercial viability.

Contribution

This paper presents the first physics-grounded economic benchmark integrating industry-standard financial data to assess autonomous navigation's real-world economic viability.

Findings

All evaluated methods yield negative contribution margins.

The best method, CANVAS, outperforms LiDAR-equipped Nav2 in economic terms.

Current navigation algorithms are not economically viable for real-world deployment.

Abstract

While current navigation benchmarks prioritize task success in simplified settings, they neglect the multidimensional economic constraints essential for the real-world commercialization of autonomous delivery systems. We introduce CostNav, an Economic Navigation Benchmark that evaluates physical AI agents through comprehensive economic cost-revenue analysis aligned with real-world business operations. By integrating industry-standard data--such as Securities and Exchange Commission (SEC) filings and Abbreviated Injury Scale (AIS) injury reports--with Isaac Sim's detailed collision and cargo dynamics, CostNav transcends simple task completion to accurately evaluate business value in complex, real-world scenarios. To our knowledge, CostNav is the first physics-grounded economic benchmark that uses industry-standard regulatory and financial data to quantitatively expose the gap between navigation research metrics and commercial viability, revealing that optimizing for task success on a simplified task fundamentally differs from optimizing for real-world economic deployment. Evaluating seven baselines--two rule-based and five imitation learning--we find that no current method is economically viable, all yielding negative contribution margins. The best-performing method, CANVAS (-27.36\$/run), equipped with only an RGB camera and GPS, outperforms LiDAR-equipped Nav2 w/ GPS (-35.46\$/run). We challenge the community to develop navigation policies that achieve economic viability on CostNav. We remain method-agnostic, evaluating success solely on cost rather than the underlying architecture. All resources are available at https://github.com/worv-ai/CostNav.