Knowledge Distillation based Contextual Relevance Matching for E-commerce Product Search

TL;DR

This paper introduces an efficient knowledge distillation framework for e-commerce relevance matching that combines Transformer and classical models, leveraging bipartite graph structures to improve accuracy and user engagement.

Contribution

It proposes a novel knowledge distillation approach with a $k$-order relevance model for improved e-commerce search relevance, addressing efficiency and structural challenges.

Findings

Significant accuracy improvement on large-scale real-world data

5.7% increase in UV-value in online A/B testing

Effective integration of Transformer and classical relevance models

Abstract

Online relevance matching is an essential task of e-commerce product search to boost the utility of search engines and ensure a smooth user experience. Previous work adopts either classical relevance matching models or Transformer-style models to address it. However, they ignore the inherent bipartite graph structures that are ubiquitous in e-commerce product search logs and are too inefficient to deploy online. In this paper, we design an efficient knowledge distillation framework for e-commerce relevance matching to integrate the respective advantages of Transformer-style models and classical relevance matching models. Especially for the core student model of the framework, we propose a novel method using $k$-order relevance modeling. The experimental results on large-scale real-world data (the size is 6$\sim$174 million) show that the proposed method significantly improves the prediction accuracy in terms of human relevance judgment. We deploy our method to the anonymous online search platform. The A/B testing results show that our method significantly improves 5.7% of UV-value under price sort mode.