New Recruiter and Jobs: The Largest Enterprise Data Migration at LinkedIn
Xie Lu, Xiaoguang Wang, Xiaoyang Gu

TL;DR
This paper details LinkedIn's large-scale data migration of core talent products onto a unified platform, focusing on architecture, challenges, and solutions to ensure data integrity and minimal downtime.
Contribution
It presents a comprehensive architecture and methodology for executing a complex enterprise data migration with minimal disruption and data discrepancies.
Findings
Successful migration with no data discrepancies
Minimal downtime achieved during migration
Framework applicable to large-scale enterprise data migrations
Abstract
In August 2019, we introduced to our members and customers the idea of moving LinkedIn's two core talent products -- Jobs and Recruiter -- onto a single platform to help talent professionals be even more productive. This single platform is called the New Recruiter & Jobs. A critical and difficult part of this effort is migrating their existing data from the legacy database to the new database and ensure there is no data discrepancy and no down time. In this article, we will discuss the general architecture for a successful data migration and the thought process we followed. Then we expand these ideas to our circumstances and explain in more detail about our specific challenges and solutions. In the Ramp Process section, we explain the inherent difficulties in satisfying our success criteria and describe how we overcome these difficulties and fulfill the success criteria practically.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBig Data and Business Intelligence · Semantic Web and Ontologies · Data Quality and Management
