Multi-Task Learning of Query Intent and Named Entities using Transfer Learning

TL;DR

This paper explores multi-task learning with transfer learning to jointly identify query intent and label task-specific named entities, improving accuracy in ambiguous scenarios like store location recognition in search queries.

Contribution

It introduces a multi-task BERT-based approach that jointly learns query intent and named entities using multiple loss functions, addressing ambiguity in task-specific NER.

Findings

Joint learning improves entity disambiguation accuracy.

Transfer learning enhances performance in task-specific NER.

Multi-task approach outperforms single-task models.

Abstract

Named entity recognition (NER) has been studied extensively and the earlier algorithms were based on sequence labeling like Hidden Markov Models (HMM) and conditional random fields (CRF). These were followed by neural network based deep learning models. Recently, BERT has shown new state of the art accuracy in sequence labeling tasks like NER. In this short article, we study various approaches to task specific NER. Task specific NER has two components - identifying the intent of a piece of text (like search queries), and then labeling the query with task specific named entities. For example, we consider the task of labeling Target store locations in a search query (which could be entered in a search box or spoken in a device like Alexa or Google Home). Store locations are highly ambiguous and sometimes it is difficult to differentiate between say a location and a non-location. For example, "pickup my order at orange store" has "orange" as the store location, while "buy orange at target" has "orange" as a fruit. We explore this difficulty by doing multi-task learning which we call global to local transfer of information. We jointly learn the query intent (i.e. store lookup) and the named entities by using multiple loss functions in our BERT based model and find interesting results.