Tuning Word2vec for Large Scale Recommendation Systems

TL;DR

This paper demonstrates that hyperparameter tuning significantly improves Word2vec's performance in large-scale recommendation systems, with methods that are efficient and scalable for real-world applications.

Contribution

It introduces constrained hyperparameter optimization techniques that enhance Word2vec's effectiveness in recommendation systems without excessive computational costs.

Findings

Unconstrained optimization improves hit rate by 221%.

Runtime-constrained optimization achieves 138% improvement.

Sampling-based hyperparameter tuning yields 91% improvement on full datasets.

Abstract

Word2vec is a powerful machine learning tool that emerged from Natural Lan-guage Processing (NLP) and is now applied in multiple domains, including recom-mender systems, forecasting, and network analysis. As Word2vec is often used offthe shelf, we address the question of whether the default hyperparameters are suit-able for recommender systems. The answer is emphatically no. In this paper, wefirst elucidate the importance of hyperparameter optimization and show that un-constrained optimization yields an average 221% improvement in hit rate over thedefault parameters. However, unconstrained optimization leads to hyperparametersettings that are very expensive and not feasible for large scale recommendationtasks. To this end, we demonstrate 138% average improvement in hit rate with aruntime budget-constrained hyperparameter optimization. Furthermore, to makehyperparameter optimization applicable for large scale recommendation problemswhere the target dataset is too large to search over, we investigate generalizinghyperparameters settings from samples. We show that applying constrained hy-perparameter optimization using only a 10% sample of the data still yields a 91%average improvement in hit rate over the default parameters when applied to thefull datasets. Finally, we apply hyperparameters learned using our method of con-strained optimization on a sample to the Who To Follow recommendation serviceat Twitter and are able to increase follow rates by 15%.