Personalized Fashion Recommendation from Personal Social Media Data: An Item-to-Set Metric Learning Approach

TL;DR

This paper introduces an item-to-set metric learning framework for personalized fashion recommendation using social media data, leveraging multi-modal feature extraction and fusion to improve recommendation accuracy.

Contribution

It proposes a novel item-to-set metric learning approach with multi-modality feature extraction for personalized fashion recommendation from social media data.

Findings

Superior performance on real-world social media dataset

Effective multi-modality feature extraction and fusion

Improved recommendation accuracy

Abstract

With the growth of online shopping for fashion products, accurate fashion recommendation has become a critical problem. Meanwhile, social networks provide an open and new data source for personalized fashion analysis. In this work, we study the problem of personalized fashion recommendation from social media data, i.e. recommending new outfits to social media users that fit their fashion preferences. To this end, we present an item-to-set metric learning framework that learns to compute the similarity between a set of historical fashion items of a user to a new fashion item. To extract features from multi-modal street-view fashion items, we propose an embedding module that performs multi-modality feature extraction and cross-modality gated fusion. To validate the effectiveness of our approach, we collect a real-world social media dataset. Extensive experiments on the collected dataset show the superior performance of our proposed approach.