Deep Natural Language Processing for LinkedIn Search Systems

TL;DR

This paper explores the application of deep NLP techniques to improve LinkedIn's search system across five tasks, addressing challenges like latency and robustness through large-scale experiments and model design insights.

Contribution

It provides a comprehensive analysis of deep NLP's effectiveness in search, offering practical solutions for latency and robustness in a commercial setting.

Findings

Deep NLP is beneficial for certain search tasks.

Latency can be effectively managed with optimized models.

Robustness issues require specific model adjustments.

Abstract

Many search systems work with large amounts of natural language data, e.g., search queries, user profiles and documents, where deep learning based natural language processing techniques (deep NLP) can be of great help. In this paper, we introduce a comprehensive study of applying deep NLP techniques to five representative tasks in search engines. Through the model design and experiments of the five tasks, readers can find answers to three important questions: (1) When is deep NLP helpful/not helpful in search systems? (2) How to address latency challenges? (3) How to ensure model robustness? This work builds on existing efforts of LinkedIn search, and is tested at scale on a commercial search engine. We believe our experiences can provide useful insights for the industry and research communities.