A Tale of Color Variants: Representation and Self-Supervised Learning in Fashion E-Commerce

TL;DR

This paper explores using self-supervised learning with color jitter augmentation to identify color variants in fashion e-commerce, achieving performance comparable to supervised methods without manual annotations.

Contribution

It demonstrates that self-supervised learning with color invariance can effectively identify fashion style variants, reducing reliance on manual annotations.

Findings

SSL methods perform comparably to supervised approaches

Color jitter augmentation enhances invariance to color variations

Proposed a novel SSL method for fashion variant recognition

Abstract

In this paper, we address a crucial problem in fashion e-commerce (with respect to customer experience, as well as revenue): color variants identification, i.e., identifying fashion products that match exactly in their design (or style), but only to differ in their color. We propose a generic framework, that leverages deep visual Representation Learning at its heart, to address this problem for our fashion e-commerce platform. Our framework could be trained with supervisory signals in the form of triplets, that are obtained manually. However, it is infeasible to obtain manual annotations for the entire huge collection of data usually present in fashion e-commerce platforms, such as ours, while capturing all the difficult corner cases. But, to our rescue, interestingly we observed that this crucial problem in fashion e-commerce could also be solved by simple color jitter based image augmentation, that recently became widely popular in the contrastive Self-Supervised Learning (SSL) literature, that seeks to learn visual representations without using manual labels. This naturally led to a question in our mind: Could we leverage SSL in our use-case, and still obtain comparable performance to our supervised framework? The answer is, Yes! because, color variant fashion objects are nothing but manifestations of a style, in different colors, and a model trained to be invariant to the color (with, or without supervision), should be able to recognize this! This is what the paper further demonstrates, both qualitatively, and quantitatively, while evaluating a couple of state-of-the-art SSL techniques, and also proposing a novel method.