WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
Shunyu Yao, Howard Chen, John Yang, Karthik Narasimhan

TL;DR
WebShop introduces a large-scale, realistic simulated e-commerce environment with real-world product data and instructions, enabling the development and evaluation of grounded language agents capable of complex web interactions and transfer to real-world sites.
Contribution
We created WebShop, a scalable, real-world inspired web environment with extensive data, and evaluated diverse agents, providing insights into language understanding and decision-making in web tasks.
Findings
Best agent achieves 29% success rate, outperforming heuristics at 9.6%.
Agents trained on WebShop show non-trivial transfer to amazon.com and ebay.com.
Analysis reveals key challenges and future directions for grounded language agents.
Abstract
Existing benchmarks for grounding language in interactive environments either lack real-world linguistic elements, or prove difficult to scale up due to substantial human involvement in the collection of data or feedback signals. To bridge this gap, we develop WebShop -- a simulated e-commerce website environment with million real-world products and crowd-sourced text instructions. Given a text instruction specifying a product requirement, an agent needs to navigate multiple types of webpages and issue diverse actions to find, customize, and purchase an item. WebShop provides several challenges for language grounding including understanding compositional instructions, query (re-)formulation, comprehending and acting on noisy text in webpages, and performing strategic exploration. We collect over human demonstrations for the task, and train and evaluate a diverse…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsTopic Modeling · Multimodal Machine Learning Applications · Natural Language Processing Techniques
