Five lessons from building a deep neural network recommender

TL;DR

This paper shares five key lessons learned from developing a deep learning-based hybrid recommender system at FINN.no, addressing challenges like data sparsity and cold-start, and demonstrating a 20% click-through rate in production.

Contribution

The paper introduces a hybrid recommender system that integrates heterogeneous marketplace data and five effective strategies for improving recommendation quality.

Findings

Staged training outperforms end-to-end training.

Leveraging diverse user behaviors enhances recommendations.

Transfer learning addresses data imbalance effectively.

Abstract

Recommendation algorithms are widely adopted in marketplaces to help users find the items they are looking for. The sparsity of the items by user matrix and the cold-start issue in marketplaces pose challenges for the off-the-shelf matrix factorization based recommender systems. To understand user intent and tailor recommendations to their needs, we use deep learning to explore various heterogeneous data available in marketplaces. This paper summarizes five lessons we learned from experimenting with state-of-the-art deep learning recommenders at the leading Norwegian marketplace FINN.no. We design a hybrid recommender system that takes the user-generated contents of a marketplace (including text, images and meta attributes) and combines them with user behavior data such as page views and messages to provide recommendations for marketplace items. Among various tactics we experimented with, the following five show the best impact: staged training instead of end-to-end training, leveraging rich user behaviors beyond page views, using user behaviors as noisy labels to train embeddings, using transfer learning to solve the unbalanced data problem, and using attention mechanisms in the hybrid model. This system is currently running with around 20% click-through-rate in production at FINN.no and serves over one million visitors everyday.