"Does it come in black?" CLIP-like models are zero-shot recommenders

TL;DR

This paper explores how CLIP-like models can enable zero-shot, attribute-based product recommendations in online fashion shopping, allowing users to find items with specific attribute changes like color.

Contribution

It introduces GradREC, a CLIP-based model tailored for fashion, demonstrating its potential for zero-shot attribute-aware recommendations and providing an initial assessment of its capabilities.

Findings

GradREC effectively suggests items with desired attribute changes.

CLIP-like models support zero-shot, attribute-specific recommendations.

Initial evaluation shows promising industry applications.

Abstract

Product discovery is a crucial component for online shopping. However, item-to-item recommendations today do not allow users to explore changes along selected dimensions: given a query item, can a model suggest something similar but in a different color? We consider item recommendations of the comparative nature (e.g. "something darker") and show how CLIP-based models can support this use case in a zero-shot manner. Leveraging a large model built for fashion, we introduce GradREC and its industry potential, and offer a first rounded assessment of its strength and weaknesses.