Multi-word Term Embeddings Improve Lexical Product Retrieval

TL;DR

This paper introduces the H1 embedding model for product search, which effectively processes multi-word product terms as single tokens, enhancing retrieval precision in e-commerce search systems.

Contribution

The paper presents a novel embedding model that handles multi-word product terms as single tokens, improving hybrid product search accuracy over existing models.

Findings

H1 model outperforms state-of-the-art models in mAP@12 and R@1k metrics

Processing multi-word terms as single tokens increases search precision

Hybrid system achieves high retrieval accuracy on WANDS dataset

Abstract

Product search is uniquely different from search for documents, Internet resources or vacancies, therefore it requires the development of specialized search systems. The present work describes the H1 embdedding model, designed for an offline term indexing of product descriptions at e-commerce platforms. The model is compared to other state-of-the-art (SoTA) embedding models within a framework of hybrid product search system that incorporates the advantages of lexical methods for product retrieval and semantic embedding-based methods. We propose an approach to building semantically rich term vocabularies for search indexes. Compared to other production semantic models, H1 paired with the proposed approach stands out due to its ability to process multi-word product terms as one token. As an example, for search queries "new balance shoes", "gloria jeans kids wear" brand entity will be represented as one token - "new balance", "gloria jeans". This results in an increased precision of the system without affecting the recall. The hybrid search system with proposed model scores mAP@12 = 56.1% and R@1k = 86.6% on the WANDS public dataset, beating other SoTA analogues.