Efficient user history modeling with amortized inference for deep learning recommendation models

TL;DR

This paper improves deep learning recommendation models by using amortized inference with a novel user history modeling approach, significantly reducing latency while maintaining recommendation quality.

Contribution

It introduces a reformulation of the M-FALCON amortized inference algorithm for DLRMs and demonstrates its effectiveness in reducing latency in real-world deployment.

Findings

Amortized inference reduces latency by 30% in LinkedIn applications.

Appending candidate items with cross-attention performs on par with concatenation.

Reformulating M-FALCON for DLRMs enables efficient user history modeling.

Abstract

We study user history modeling via Transformer encoders in deep learning recommendation models (DLRM). Such architectures can significantly improve recommendation quality, but usually incur high latency cost necessitating infrastructure upgrades or very small Transformer models. An important part of user history modeling is early fusion of the candidate item and various methods have been studied. We revisit early fusion and compare concatenation of the candidate to each history item against appending it to the end of the list as a separate item. Using the latter method, allows us to reformulate the recently proposed amortized history inference algorithm M-FALCON \cite{zhai2024actions} for the case of DLRM models. We show via experimental results that appending with cross-attention performs on par with concatenation and that amortization significantly reduces inference costs. We conclude with results from deploying this model on the LinkedIn Feed and Ads surfaces, where amortization reduces latency by 30\% compared to non-amortized inference.