Continuous Prompt Tuning Based Textual Entailment Model for E-commerce Entity Typing

TL;DR

This paper introduces a novel e-commerce entity typing approach that reformulates the task as textual entailment, utilizing continuous prompt tuning and fusion embeddings to better handle new entities and domain-specific language styles, achieving improved performance.

Contribution

The paper presents a continuous prompt tuning method for generating textual entailment hypotheses and fusion embeddings to adapt entity typing to e-commerce domain-specific language styles.

Findings

Improves average F1 score by around 2% over baseline.

Reformulates entity typing as textual entailment for better handling of new entities.

Demonstrates effectiveness of fusion embeddings and continuous prompt tuning.

Abstract

The explosion of e-commerce has caused the need for processing and analysis of product titles, like entity typing in product titles. However, the rapid activity in e-commerce has led to the rapid emergence of new entities, which is difficult to be solved by general entity typing. Besides, product titles in e-commerce have very different language styles from text data in general domain. In order to handle new entities in product titles and address the special language styles problem of product titles in e-commerce domain, we propose our textual entailment model with continuous prompt tuning based hypotheses and fusion embeddings for e-commerce entity typing. First, we reformulate the entity typing task into a textual entailment problem to handle new entities that are not present during training. Second, we design a model to automatically generate textual entailment hypotheses using a continuous prompt tuning method, which can generate better textual entailment hypotheses without manual design. Third, we utilize the fusion embeddings of BERT embedding and CharacterBERT embedding with a two-layer MLP classifier to solve the problem that the language styles of product titles in e-commerce are different from that of general domain. To analyze the effect of each contribution, we compare the performance of entity typing and textual entailment model, and conduct ablation studies on continuous prompt tuning and fusion embeddings. We also evaluate the impact of different prompt template initialization for the continuous prompt tuning. We show our proposed model improves the average F1 score by around 2% compared to the baseline BERT entity typing model.