Adversarial Training of Word2Vec for Basket Completion

TL;DR

This paper introduces Adversarial Negative Sampling, a GAN-inspired method to enhance Word2Vec training for basket completion, leading to significant performance improvements over standard loss functions.

Contribution

It proposes a novel adversarial training approach for Word2Vec, leveraging GAN ideas to improve negative sampling in recommendation tasks.

Findings

Significant performance improvements in basket completion tasks.

Effective stabilization of GAN training with discrete data.

Outperforms standard Negative Sampling and Noise Contrastive Estimation.

Abstract

In recent years, the Word2Vec model trained with the Negative Sampling loss function has shown state-of-the-art results in a number of machine learning tasks, including language modeling tasks, such as word analogy and word similarity, and in recommendation tasks, through Prod2Vec, an extension that applies to modeling user shopping activity and user preferences. Several methods that aim to improve upon the standard Negative Sampling loss have been proposed. In our paper we pursue more sophisticated Negative Sampling, by leveraging ideas from the field of Generative Adversarial Networks (GANs), and propose Adversarial Negative Sampling. We build upon the recent progress made in stabilizing the training objective of GANs in the discrete data setting, and introduce a new GAN-Word2Vec model.We evaluate our model on the task of basket completion, and show significant improvements in performance over Word2Vec trained using standard loss functions, including Noise Contrastive Estimation and Negative Sampling.