New Recruiter and Jobs: The Largest Enterprise Data Migration at LinkedIn

TL;DR

This paper details LinkedIn's large-scale data migration of core talent products onto a unified platform, focusing on architecture, challenges, and solutions to ensure data integrity and minimal downtime.

Contribution

It presents a comprehensive architecture and methodology for executing a complex enterprise data migration with minimal disruption and data discrepancies.

Findings

Successful migration with no data discrepancies

Minimal downtime achieved during migration

Framework applicable to large-scale enterprise data migrations

Abstract

In August 2019, we introduced to our members and customers the idea of moving LinkedIn's two core talent products -- Jobs and Recruiter -- onto a single platform to help talent professionals be even more productive. This single platform is called the New Recruiter & Jobs. A critical and difficult part of this effort is migrating their existing data from the legacy database to the new database and ensure there is no data discrepancy and no down time. In this article, we will discuss the general architecture for a successful data migration and the thought process we followed. Then we expand these ideas to our circumstances and explain in more detail about our specific challenges and solutions. In the Ramp Process section, we explain the inherent difficulties in satisfying our success criteria and describe how we overcome these difficulties and fulfill the success criteria practically.