Telepath: Understanding Users from a Human Vision Perspective in Large-Scale Recommender Systems

TL;DR

Telepath is a vision-inspired recommender system that models user understanding by mimicking human visual and cognitive processes, significantly improving key performance metrics in JD's large-scale e-commerce platform.

Contribution

The paper introduces Telepath, a novel vision-based neural network model combining CNN, RNN, and DNN to better understand user preferences from visual signals, enhancing recommendation effectiveness.

Findings

CTR increased by 1.59% in recommendation blocks.

GMV increased by 8.16% in recommendation blocks.

Advertising ROI increased by over 65% after deployment.

Abstract

Designing an e-commerce recommender system that serves hundreds of millions of active users is a daunting challenge. From a human vision perspective, there're two key factors that affect users' behaviors: items' attractiveness and their matching degree with users' interests. This paper proposes Telepath, a vision-based bionic recommender system model, which understands users from such perspective. Telepath is a combination of a convolutional neural network (CNN), a recurrent neural network (RNN) and deep neural networks (DNNs). Its CNN subnetwork simulates the human vision system to extract key visual signals of items' attractiveness and generate corresponding activations. Its RNN and DNN subnetworks simulate cerebral cortex to understand users' interest based on the activations generated from browsed items. In practice, the Telepath model has been launched to JD's recommender system and advertising system. For one of the major item recommendation blocks on the JD app, click-through rate (CTR), gross merchandise value (GMV) and orders have increased 1.59%, 8.16% and 8.71% respectively. For several major ads publishers of JD demand-side platform, CTR, GMV and return on investment have increased 6.58%, 61.72% and 65.57% respectively by the first launch, and further increased 2.95%, 41.75% and 41.37% respectively by the second launch.