Learning Fashion Compatibility from In-the-wild Images

TL;DR

This paper introduces a self-supervised learning approach for fashion compatibility prediction using in-the-wild street images, reducing reliance on labeled datasets and improving cross-dataset performance.

Contribution

It proposes a novel self-supervised method leveraging co-occurrence of items on the same person and an adversarial loss to bridge domain gaps between in-the-wild and catalog images.

Findings

Outperforms existing self-supervised methods on fashion compatibility benchmarks.

Significant improvements in cross-dataset compatibility prediction.

Effective domain gap reduction between in-the-wild and catalog images.

Abstract

Complementary fashion recommendation aims at identifying items from different categories (e.g. shirt, footwear, etc.) that "go well together" as an outfit. Most existing approaches learn representation for this task using labeled outfit datasets containing manually curated compatible item combinations. In this work, we propose to learn representations for compatibility prediction from in-the-wild street fashion images through self-supervised learning by leveraging the fact that people often wear compatible outfits. Our pretext task is formulated such that the representations of different items worn by the same person are closer compared to those worn by other people. Additionally, to reduce the domain gap between in-the-wild and catalog images during inference, we introduce an adversarial loss that minimizes the difference in feature distribution between the two domains. We conduct our experiments on two popular fashion compatibility benchmarks - Polyvore and Polyvore-Disjoint outfits, and outperform existing self-supervised approaches, particularly significant in cross-dataset setting where training and testing images are from different sources.