Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligence
Yining Hong, Rui Sun, Bingxuan Li, Xingcheng Yao, Maxine Wu, Alexander Chien, Da Yin, Ying Nian Wu, Zhecan James Wang, Kai-Wei Chang

TL;DR
This paper introduces Embodied Web Agents, a new AI paradigm that integrates physical interaction and web reasoning, supported by a unified simulation platform and benchmark for diverse tasks like navigation and cooking.
Contribution
The paper presents a novel embodied web agent framework, a comprehensive simulation environment, and a benchmark suite for evaluating integrated physical and digital intelligence in AI agents.
Findings
Significant performance gaps between AI systems and humans on integrated tasks.
The benchmark reveals challenges in current AI's embodied and web reasoning capabilities.
Opportunities identified for advancing embodied cognition and web knowledge integration.
Abstract
AI agents today are mostly siloed - they either retrieve and reason over vast amount of digital information and knowledge obtained online; or interact with the physical world through embodied perception, planning and action - but rarely both. This separation limits their ability to solve tasks that require integrated physical and digital intelligence, such as cooking from online recipes, navigating with dynamic map data, or interpreting real-world landmarks using web knowledge. We introduce Embodied Web Agents, a novel paradigm for AI agents that fluidly bridge embodiment and web-scale reasoning. To operationalize this concept, we first develop the Embodied Web Agents task environments, a unified simulation platform that tightly integrates realistic 3D indoor and outdoor environments with functional web interfaces. Building upon this platform, we construct and release the Embodied Web…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSpatial Cognition and Navigation · Multimodal Machine Learning Applications · Social Robot Interaction and HRI
