Bootstrapping Complete The Look at Pinterest

TL;DR

This paper presents Pinterest's Complete The Look system, which learns style compatibility to recommend complementary fashion items, using a large dataset and a scalable pipeline, marking a significant step in industrial-scale fashion recommendation.

Contribution

We introduce an industrial-scale system for style compatibility, including a large outfit dataset, evaluation methods, and insights from deployment in production.

Findings

Successful recommendation of compatible fashion items across categories.

Development of a scalable pipeline for dataset collection and refresh.

Insights into mitigating failure modes in production environment.

Abstract

Putting together an ideal outfit is a process that involves creativity and style intuition. This makes it a particularly difficult task to automate. Existing styling products generally involve human specialists and a highly curated set of fashion items. In this paper, we will describe how we bootstrapped the Complete The Look (CTL) system at Pinterest. This is a technology that aims to learn the subjective task of "style compatibility" in order to recommend complementary items that complete an outfit. In particular, we want to show recommendations from other categories that are compatible with an item of interest. For example, what are some heels that go well with this cocktail dress? We will introduce our outfit dataset of over 1 million outfits and 4 million objects, a subset of which we will make available to the research community, and describe the pipeline used to obtain and refresh this dataset. Furthermore, we will describe how we evaluate this subjective task and compare model performance across multiple training methods. Lastly, we will share our lessons going from experimentation to working prototype, and how to mitigate failure modes in the production environment. Our work represents one of the first examples of an industrial-scale solution for compatibility-based fashion recommendation.