Query Rewriting via Cycle-Consistent Translation for E-Commerce Search

TL;DR

This paper introduces a cycle-consistent neural network approach for query rewriting in e-commerce search, improving semantic matching and retrieval accuracy by leveraging click log data and optimizing for industrial deployment.

Contribution

It formulates query rewriting as a cyclic translation problem and develops a novel training algorithm, significantly enhancing diversity and relevance over rule-based methods.

Findings

Improved query rewriting accuracy and diversity.

Significant online business metric improvements.

Deployed at scale serving hundreds of millions of users.

Abstract

Nowadays e-commerce search has become an integral part of many people's shopping routines. One critical challenge in today's e-commerce search is the semantic matching problem where the relevant items may not contain the exact terms in the user query. In this paper, we propose a novel deep neural network based approach to query rewriting, in order to tackle this problem. Specifically, we formulate query rewriting into a cyclic machine translation problem to leverage abundant click log data. Then we introduce a novel cyclic consistent training algorithm in conjunction with state-of-the-art machine translation models to achieve the optimal performance in terms of query rewriting accuracy. In order to make it practical in industrial scenarios, we optimize the syntax tree construction to reduce computational cost and online serving latency. Offline experiments show that the proposed method is able to rewrite hard user queries into more standard queries that are more appropriate for the inverted index to retrieve. Comparing with human curated rule-based method, the proposed model significantly improves query rewriting diversity while maintaining good relevancy. Online A/B experiments show that it improves core e-commerce business metrics significantly. Since the summer of 2020, the proposed model has been launched into our search engine production, serving hundreds of millions of users.