CRAFT: Complementary Recommendations Using Adversarial Feature Transformer

TL;DR

This paper introduces CRAFT, an unsupervised adversarial transformer framework that generates diverse visual recommendations for complementary fashion items, outperforming baseline methods in human evaluations.

Contribution

CRAFT is the first to use adversarial feature transformers for visual complementary item recommendation, enabling diverse and realistic suggestions without labeled data.

Findings

Recommendations are diverse and human-preferred.

The model effectively learns cross-category visual relationships.

Generated samples closely match real-world examples.

Abstract

Traditional approaches for complementary product recommendations rely on behavioral and non-visual data such as customer co-views or co-buys. However, certain domains such as fashion are primarily visual. We propose a framework that harnesses visual cues in an unsupervised manner to learn the distribution of co-occurring complementary items in real world images. Our model learns a non-linear transformation between the two manifolds of source and target complementary item categories (e.g., tops and bottoms in outfits). Given a large dataset of images containing instances of co-occurring object categories, we train a generative transformer network directly on the feature representation space by casting it as an adversarial optimization problem. Such a conditional generative model can produce multiple novel samples of complementary items (in the feature space) for a given query item. The final recommendations are selected from the closest real world examples to the synthesized complementary features. We apply our framework to the task of recommending complementary tops for a given bottom clothing item. The recommendations made by our system are diverse, and are favored by human experts over the baseline approaches.