MaRI: Accelerating Ranking Model Inference via Structural Re-parameterization in Large Scale Recommendation System

TL;DR

MaRI introduces a structural re-parameterization method that accelerates large-scale ranking model inference in recommendation systems without sacrificing accuracy by reducing redundant computations.

Contribution

This paper presents MaRI, a novel framework that leverages structural re-parameterization to achieve lossless acceleration of ranking models in recommendation systems.

Findings

Achieves inference acceleration without accuracy loss.

Reduces redundant user-side computations in feature fusion.

Complementary to existing lightweighting and distillation methods.

Abstract

Ranking models, i.e., coarse-ranking and fine-ranking models, serve as core components in large-scale recommendation systems, responsible for scoring massive item candidates based on user preferences. To meet the stringent latency requirements of online serving, structural lightweighting or knowledge distillation techniques are commonly employed for ranking model acceleration. However, these approaches typically lead to a non-negligible drop in accuracy. Notably, the angle of lossless acceleration by optimizing feature fusion matrix multiplication, particularly through structural reparameterization, remains underexplored. In this paper, we propose MaRI, a novel Matrix Re-parameterized Inference framework, which serves as a complementary approach to existing techniques while accelerating ranking model inference without any accuracy loss. MaRI is motivated by the observation that user-side computation is redundant in feature fusion matrix multiplication, and we therefore adopt the philosophy of structural reparameterization to alleviate such redundancy.