Towards Better Query Classification with Multi-Expert Knowledge Condensation in JD Ads Search

TL;DR

This paper introduces a knowledge condensation framework that enhances FastText query classification in JD Ads Search by leveraging BERT for data retrieval and multi-expert learning, balancing accuracy and efficiency.

Contribution

The paper proposes a novel knowledge distillation method combining BERT-based data retrieval and multi-expert learning to improve FastText classification performance under low latency constraints.

Findings

Improved classification accuracy on low-frequency queries.

Effective data augmentation using BERT for better training.

Validated through offline and online experiments in JD search.

Abstract

Search query classification, as an effective way to understand user intents, is of great importance in real-world online ads systems. To ensure a lower latency, a shallow model (e.g. FastText) is widely used for efficient online inference. However, the representation ability of the FastText model is insufficient, resulting in poor classification performance, especially on some low-frequency queries and tailed categories. Using a deeper and more complex model (e.g. BERT) is an effective solution, but it will cause a higher online inference latency and more expensive computing costs. Thus, how to juggle both inference efficiency and classification performance is obviously of great practical importance. To overcome this challenge, in this paper, we propose knowledge condensation (KC), a simple yet effective knowledge distillation framework to boost the classification performance of the online FastText model under strict low latency constraints. Specifically, we propose to train an offline BERT model to retrieve more potentially relevant data. Benefiting from its powerful semantic representation, more relevant labels not exposed in the historical data will be added into the training set for better FastText model training. Moreover, a novel distribution-diverse multi-expert learning strategy is proposed to further improve the mining ability of relevant data. By training multiple BERT models from different data distributions, it can respectively perform better at high, middle, and low-frequency search queries. The model ensemble from multi-distribution makes its retrieval ability more powerful. We have deployed two versions of this framework in JD search, and both offline experiments and online A/B testing from multiple datasets have validated the effectiveness of the proposed approach.