Diversity in Fashion Recommendation using Semantic Parsing

TL;DR

This paper introduces a part-based similarity approach for fashion recommendation, leveraging visual attention and texture encoding to improve retrieval accuracy and provide explicit explanations of similarities.

Contribution

It proposes a novel part-based similarity learning method using weakly-supervised data, visual attention, and texture encoding, surpassing state-of-the-art in fashion retrieval.

Findings

Outperforms existing methods on DeepFashion dataset

Enables explicit part-based similarity explanations

Improves retrieval accuracy in fashion recommendation

Abstract

Developing recommendation system for fashion images is challenging due to the inherent ambiguity associated with what criterion a user is looking at. Suggesting multiple images where each output image is similar to the query image on the basis of a different feature or part is one way to mitigate the problem. Existing works for fashion recommendation have used Siamese or Triplet network to learn features between a similar pair and a similar-dissimilar triplet respectively. However, these methods do not provide basic information such as, how two clothing images are similar, or which parts present in the two images make them similar. In this paper, we propose to recommend images by explicitly learning and exploiting part based similarity. We propose a novel approach of learning discriminative features from weakly-supervised data by using visual attention over the parts and a texture encoding network. We show that the learned features surpass the state-of-the-art in retrieval task on DeepFashion dataset. We then use the proposed model to recommend fashion images having an explicit variation with respect to similarity of any of the parts.