Accelerating E-Commerce Search Engine Ranking by Contextual Factor Selection

TL;DR

This paper introduces Contextual Factor Selection (CFS), a reinforcement learning-based method to select effective factors for search ranking, significantly improving efficiency and response latency in large-scale e-commerce search engines like Taobao.

Contribution

It proposes a novel CFS approach that dynamically selects factors for each search instance, balancing ranking quality and system efficiency, with demonstrated real-world benefits.

Findings

Outperforms existing feature selection methods offline.

Reduces service latency significantly in real-world tests.

Improves search engine responsiveness during peak shopping events.

Abstract

In industrial large-scale search systems, such as Taobao.com search for commodities, the quality of the ranking result is getting continually improved by introducing more factors from complex procedures, e.g., deep neural networks for extracting image factors. Meanwhile, the increasing of the factors demands more computation resource and raises the system response latency. It has been observed that a search instance usually requires only a small set of effective factors, instead of all factors. Therefore, removing ineffective factors significantly improves the system efficiency. This paper studies the \emph{Contextual Factor Selection} (CFS), which selects only a subset of effective factors for every search instance, for a well balance between the search quality and the response latency. We inject CFS into the search engine ranking score to accelerate the engine, considering both ranking effectiveness and efficiency. The learning of the CFS model involves a combinatorial optimization, which is transformed as a sequential decision-making problem. Solving the problem by reinforcement learning, we propose the RankCFS, which has been assessed in an off-line environment as well as a real-world on-line environment (Taobao.com). The empirical results show that, the proposed CFS approach outperforms several existing supervised/unsupervised methods for feature selection in the off-line environment, and also achieves significant real-world performance improvement, in term of service latency, in daily test as well as Singles' Day Shopping Festival in $2017$.