Semi-Supervised Visual Representation Learning for Fashion Compatibility

TL;DR

This paper introduces a semi-supervised learning method for fashion compatibility prediction that leverages unlabeled data and consistency regularization, achieving comparable results to fully supervised approaches.

Contribution

It proposes a novel semi-supervised framework using pseudo-outfits and self-supervision for fashion compatibility prediction, reducing reliance on labeled data.

Findings

Achieves comparable performance to fully supervised methods with less labeled data.

Effectively utilizes unlabeled fashion data through pseudo-outfits.

Demonstrates robustness across multiple large-scale datasets.

Abstract

We consider the problem of complementary fashion prediction. Existing approaches focus on learning an embedding space where fashion items from different categories that are visually compatible are closer to each other. However, creating such labeled outfits is intensive and also not feasible to generate all possible outfit combinations, especially with large fashion catalogs. In this work, we propose a semi-supervised learning approach where we leverage large unlabeled fashion corpus to create pseudo-positive and pseudo-negative outfits on the fly during training. For each labeled outfit in a training batch, we obtain a pseudo-outfit by matching each item in the labeled outfit with unlabeled items. Additionally, we introduce consistency regularization to ensure that representation of the original images and their transformations are consistent to implicitly incorporate colour and other important attributes through self-supervision. We conduct extensive experiments on Polyvore, Polyvore-D and our newly created large-scale Fashion Outfits datasets, and show that our approach with only a fraction of labeled examples performs on-par with completely supervised methods.