Extracting Important Tokens in E-Commerce Queries with a Tag Interaction-Aware Transformer Model

TL;DR

This paper introduces TagBERT, a transformer-based model that leverages semantic tags for token classification to improve query reformulation in e-commerce search, outperforming existing models.

Contribution

The paper proposes a novel dependency-aware transformer model, TagBERT, which utilizes semantic token tags to enhance query reformulation accuracy in e-commerce search.

Findings

TagBERT outperforms BERT, eBERT, and sequence-to-sequence models.

Semantic tags significantly improve token classification.

Model demonstrates superior performance on large real-world datasets.

Abstract

The major task of any e-commerce search engine is to retrieve the most relevant inventory items, which best match the user intent reflected in a query. This task is non-trivial due to many reasons, including ambiguous queries, misaligned vocabulary between buyers, and sellers, over- or under-constrained queries by the presence of too many or too few tokens. To address these challenges, query reformulation is used, which modifies a user query through token dropping, replacement or expansion, with the objective to bridge semantic gap between query tokens and users' search intent. Early methods of query reformulation mostly used statistical measures derived from token co-occurrence frequencies from selective user sessions having clicks or purchases. In recent years, supervised deep learning approaches, specifically transformer-based neural language models, or sequence-to-sequence models are being used for query reformulation task. However, these models do not utilize the semantic tags of a query token, which are significant for capturing user intent of an e-commerce query. In this work, we pose query reformulation as a token classification task, and solve this task by designing a dependency-aware transformer-based language model, TagBERT, which makes use of semantic tags of a token for learning superior query phrase embedding. Experiments on large, real-life e-commerce datasets show that TagBERT exhibits superior performance than plethora of competing models, including BERT, eBERT, and Sequence-to-Sequence transformer model for important token classification task.