From Features to Transformers: Redefining Ranking for Scalable Impact

TL;DR

LiGR introduces a scalable transformer-based ranking framework that reduces manual feature engineering, improves performance with larger models and data, and enhances diversity through set-wise scoring, suitable for production at LinkedIn.

Contribution

The paper presents a novel transformer architecture with learned normalization and set-wise attention, enabling scalable, high-performance ranking with minimal feature engineering.

Findings

Outperforms previous systems with fewer features

Validates scaling law for ranking models

Improves diversity through set-wise scoring

Abstract

We present LiGR, a large-scale ranking framework developed at LinkedIn that brings state-of-the-art transformer-based modeling architectures into production. We introduce a modified transformer architecture that incorporates learned normalization and simultaneous set-wise attention to user history and ranked items. This architecture enables several breakthrough achievements, including: (1) the deprecation of most manually designed feature engineering, outperforming the prior state-of-the-art system using only few features (compared to hundreds in the baseline), (2) validation of the scaling law for ranking systems, showing improved performance with larger models, more training data, and longer context sequences, and (3) simultaneous joint scoring of items in a set-wise manner, leading to automated improvements in diversity. To enable efficient serving of large ranking models, we describe techniques to scale inference effectively using single-pass processing of user history and set-wise attention. We also summarize key insights from various ablation studies and A/B tests, highlighting the most impactful technical approaches.