An End-to-End Solution for Named Entity Recognition in eCommerce Search

TL;DR

This paper presents an end-to-end NER solution for eCommerce search using a novel TripleLearn training framework that significantly improves model performance and user engagement.

Contribution

The paper introduces TripleLearn, a new training framework that learns from three datasets simultaneously, enhancing NER accuracy in eCommerce search.

Findings

F1 score improved from 69.5 to 93.3 with TripleLearn.

Online A/B tests show increased user engagement and revenue.

Model has been operational on homedepot.com for over 9 months.

Abstract

Named entity recognition (NER) is a critical step in modern search query understanding. In the domain of eCommerce, identifying the key entities, such as brand and product type, can help a search engine retrieve relevant products and therefore offer an engaging shopping experience. Recent research shows promising results on shared benchmark NER tasks using deep learning methods, but there are still unique challenges in the industry regarding domain knowledge, training data, and model production. This paper demonstrates an end-to-end solution to address these challenges. The core of our solution is a novel model training framework "TripleLearn" which iteratively learns from three separate training datasets, instead of one training set as is traditionally done. Using this approach, the best model lifts the F1 score from 69.5 to 93.3 on the holdout test data. In our offline experiments, TripleLearn improved the model performance compared to traditional training approaches which use a single set of training data. Moreover, in the online A/B test, we see significant improvements in user engagement and revenue conversion. The model has been live on homedepot.com for more than 9 months, boosting search conversions and revenue. Beyond our application, this TripleLearn framework, as well as the end-to-end process, is model-independent and problem-independent, so it can be generalized to more industrial applications, especially to the eCommerce industry which has similar data foundations and problems.