Distant Supervision for E-commerce Query Segmentation via Attention Network

TL;DR

This paper introduces a novel attention-based BiLSTM-CRF model leveraging distant supervision and external document contexts to improve e-commerce query segmentation, addressing data scarcity and OOV issues.

Contribution

It proposes a new method that uses external document contexts with an attention mechanism to enhance deep learning-based query segmentation in e-commerce.

Findings

Outperforms several baseline methods on two datasets.

Effectively utilizes external document contexts for better segmentation.

Addresses OOV and data scarcity challenges in e-commerce query segmentation.

Abstract

The booming online e-commerce platforms demand highly accurate approaches to segment queries that carry the product requirements of consumers. Recent works have shown that the supervised methods, especially those based on deep learning, are attractive for achieving better performance on the problem of query segmentation. However, the lack of labeled data is still a big challenge for training a deep segmentation network, and the problem of Out-of-Vocabulary (OOV) also adversely impacts the performance of query segmentation. Different from query segmentation task in an open domain, e-commerce scenario can provide external documents that are closely related to these queries. Thus, to deal with the two challenges, we employ the idea of distant supervision and design a novel method to find contexts in external documents and extract features from these contexts. In this work, we propose a BiLSTM-CRF based model with an attention module to encode external features, such that external contexts information, which can be utilized naturally and effectively to help query segmentation. Experiments on two datasets show the effectiveness of our approach compared with several kinds of baselines.