Visually Aware Skip-Gram for Image Based Recommendations

TL;DR

This paper introduces VASG, a novel framework combining Skip-Gram and neural networks to learn user and product representations from images, enabling personalized recommendations including for cold-start products.

Contribution

The paper presents VASG, a new end-to-end model that integrates visual features with user-product co-occurrence data for improved recommendations.

Findings

Effective recommendations using nearest neighbor search.

Enables cold-start product recommendations.

Outperforms baseline models on real-world datasets.

Abstract

The visual appearance of a product significantly influences purchase decisions on e-commerce websites. We propose a novel framework VASG (Visually Aware Skip-Gram) for learning user and product representations in a common latent space using product image features. Our model is an amalgamation of the Skip-Gram architecture and a deep neural network based Decoder. Here the Skip-Gram attempts to capture user preference by optimizing user-product co-occurrence in a Heterogeneous Information Network while the Decoder simultaneously learns a mapping to transform product image features to the Skip-Gram embedding space. This architecture is jointly optimized in an end-to-end, multitask fashion. The proposed framework enables us to make personalized recommendations for cold-start products which have no purchase history. Experiments conducted on large real-world datasets show that the learned embeddings can generate effective recommendations using nearest neighbour searches.