Semi-supervised Fashion Compatibility Prediction by Color Distortion Prediction

TL;DR

This paper introduces a semi-supervised approach for fashion compatibility prediction by using a novel color distortion prediction task, which enhances feature learning without extensive labeled data, addressing the challenges of fast-changing fashion trends.

Contribution

It proposes a new color distortion prediction pretext task that improves feature extraction in fashion compatibility models, reducing reliance on large labeled datasets.

Findings

The proposed method outperforms baseline models in compatibility prediction.

Applying the color distortion task improves feature representation quality.

The approach is effective across multiple state-of-the-art methods.

Abstract

Supervised learning methods have been suffering from the fact that a large-scale labeled dataset is mandatory, which is difficult to obtain. This has been a more significant issue for fashion compatibility prediction because compatibility aims to capture people's perception of aesthetics, which are sparse and changing. Thus, the labeled dataset may become outdated quickly due to fast fashion. Moreover, labeling the dataset always needs some expert knowledge; at least they should have a good sense of aesthetics. However, there are limited self/semi-supervised learning techniques in this field. In this paper, we propose a general color distortion prediction task forcing the baseline to recognize low-level image information to learn more discriminative representation for fashion compatibility prediction. Specifically, we first propose to distort the image by adjusting the image color balance, contrast, sharpness, and brightness. Then, we propose adding Gaussian noise to the distorted image before passing them to the convolutional neural network (CNN) backbone to learn a probability distribution over all possible distortions. The proposed pretext task is adopted in the state-of-the-art methods in fashion compatibility and shows its effectiveness in improving these methods' ability in extracting better feature representations. Applying the proposed pretext task to the baseline can consistently outperform the original baseline.