Deep Natural Language Processing for LinkedIn Search

TL;DR

This paper explores the application of deep NLP techniques, including BERT pre-training, to improve various search system tasks at LinkedIn, addressing challenges like latency and robustness.

Contribution

It provides a comprehensive study of deep NLP methods applied to five key search tasks and introduces BERT pre-training as a sixth versatile task.

Findings

Deep NLP improves query intent prediction and document ranking.

BERT pre-training enhances multiple search tasks.

Insights on latency and robustness challenges in large-scale deployment.

Abstract

Many search systems work with large amounts of natural language data, e.g., search queries, user profiles, and documents. Building a successful search system requires a thorough understanding of textual data semantics, where deep learning based natural language processing techniques (deep NLP) can be of great help. In this paper, we introduce a comprehensive study for applying deep NLP techniques to five representative tasks in search systems: query intent prediction (classification), query tagging (sequential tagging), document ranking (ranking), query auto completion (language modeling), and query suggestion (sequence to sequence). We also introduce BERT pre-training as a sixth task that can be applied to many of the other tasks. Through the model design and experiments of the six tasks, readers can find answers to four important questions: (1). When is deep NLP helpful/not helpful in search systems? (2). How to address latency challenges? (3). How to ensure model robustness? This work builds on existing efforts of LinkedIn search, and is tested at scale on LinkedIn's commercial search engines. We believe our experiences can provide useful insights for the industry and research communities.