Shop The Look: Building a Large Scale Visual Shopping System at Pinterest
Raymond Shiau, Hao-Yu Wu, Eric Kim, Yue Li Du, Anqi Guo, Zhiyuan, Zhang, Eileen Li, Kunlong Gu, Charles Rosenberg, Andrew Zhai

TL;DR
This paper describes the development and deployment of 'Shop The Look,' a large-scale visual shopping system at Pinterest that enhances product discovery through visual search, achieving significant improvements in relevance and engagement.
Contribution
The paper presents a comprehensive system for visual shopping at scale, including technology, infrastructure, and data labeling strategies, with real-world deployment and measurable impact.
Findings
Over 160% improvement in human relevance judgments
Over 80% increase in user engagement
Successful deployment in Pinterest production environment
Abstract
As online content becomes ever more visual, the demand for searching by visual queries grows correspondingly stronger. Shop The Look is an online shopping discovery service at Pinterest, leveraging visual search to enable users to find and buy products within an image. In this work, we provide a holistic view of how we built Shop The Look, a shopping oriented visual search system, along with lessons learned from addressing shopping needs. We discuss topics including core technology across object detection and visual embeddings, serving infrastructure for realtime inference, and data labeling methodology for training/evaluation data collection and human evaluation. The user-facing impacts of our system design choices are measured through offline evaluations, human relevance judgements, and online A/B experiments. The collective improvements amount to cumulative relative gains of over…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
