Searching, fast and slow, through product catalogs

TL;DR

This paper introduces a unified SKU search architecture combining real-time suggestions and low-latency search, outperforming default engines and enhancing descriptions with generative models for improved user experience.

Contribution

It presents a novel integrated SKU search system utilizing Trie, TF-IDF, and language models, with ablation studies and enhancements via GPT-3.5-turbo.

Findings

System vastly outperforms default search engines in SKU retrieval.

Combines multiple components to balance speed and accuracy.

Enhances SKU descriptions with generative text models for better user context.

Abstract

String matching algorithms in the presence of abbreviations, such as in Stock Keeping Unit (SKU) product catalogs, remains a relatively unexplored topic. In this paper, we present a unified architecture for SKU search that provides both a real-time suggestion system (based on a Trie data structure) as well as a lower latency search system (making use of character level TF-IDF in combination with language model vector embeddings) where users initiate the search process explicitly. We carry out ablation studies that justify designing a complex search system composed of multiple components to address the delicate trade-off between speed and accuracy. Using SKU search in the Dynamics CRM as an example, we show how our system vastly outperforms, in all aspects, the results provided by the default search engine. Finally, we show how SKU descriptions may be enhanced via generative text models (using gpt-3.5-turbo) so that the consumers of the search results may get more context and a generally better experience when presented with the results of their SKU search.