AgenticShop: Benchmarking Agentic Product Curation for Personalized Web Shopping

TL;DR

AgenticShop introduces a comprehensive benchmark for evaluating personalized agentic product curation in open-web shopping environments, highlighting current system limitations and emphasizing the need for improved user-centric solutions.

Contribution

This paper presents the first benchmark for assessing agentic systems in open-web shopping, incorporating realistic scenarios, diverse user profiles, and personalized evaluation methods.

Findings

Current agentic systems are largely insufficient for personalized product curation.

The benchmark reveals significant gaps in existing systems' ability to adapt to user preferences.

AgenticShop provides a new standard for evaluating agentic shopping systems.

Abstract

The proliferation of e-commerce has made web shopping platforms key gateways for customers navigating the vast digital marketplace. Yet this rapid expansion has led to a noisy and fragmented information environment, increasing cognitive burden as shoppers explore and purchase products online. With promising potential to alleviate this challenge, agentic systems have garnered growing attention for automating user-side tasks in web shopping. Despite significant advancements, existing benchmarks fail to comprehensively evaluate how well agentic systems can curate products in open-web settings. Specifically, they have limited coverage of shopping scenarios, focusing only on simplified single-platform lookups rather than exploratory search. Moreover, they overlook personalization in evaluation, leaving unclear whether agents can adapt to diverse user preferences in realistic shopping contexts. To address this gap, we present AgenticShop, the first benchmark for evaluating agentic systems on personalized product curation in open-web environment. Crucially, our approach features realistic shopping scenarios, diverse user profiles, and a verifiable, checklist-driven personalization evaluation framework. Through extensive experiments, we demonstrate that current agentic systems remain largely insufficient, emphasizing the need for user-side systems that effectively curate tailored products across the modern web.