A Light-speed Linear Program Solver for Personalized Recommendation with Diversity Constraints

TL;DR

This paper introduces a specialized linear program solver tailored for real-time personalized recommendation systems with diversity constraints, achieving solutions within 1 millisecond and significantly outperforming existing solvers.

Contribution

We develop a novel, fast LP solver that exploits problem structure and dual problem properties to meet real-time latency requirements in recommender systems.

Findings

Solver achieves ~1 ms runtime, 20x faster than standard solvers.

Method maintains recommendation quality while reducing latency.

Screening technique effectively reduces problem dimensionality during optimization.

Abstract

We study a structured linear program (LP) that emerges in the need of ranking candidates or items in personalized recommender systems. Since the candidate set is only known in real time, the LP also needs to be formed and solved in real time. Latency and user experience are major considerations, requiring the LP to be solved within just a few milliseconds. Although typical instances of the problem are not very large in size, this stringent time limit appears to be beyond the capability of most existing (commercial) LP solvers, which can take $20$ milliseconds or more to find a solution. Thus, reliable methods that address the real-world complication of latency become necessary. In this paper, we propose a fast specialized LP solver for a structured problem with diversity constraints. Our method solves the dual problem, making use of the piece-wise affine structure of the dual objective function, with an additional screening technique that helps reduce the dimensionality of the problem as the algorithm progresses. Experiments reveal that our method can solve the problem within roughly 1 millisecond, yielding a 20x improvement in speed over efficient off-the-shelf LP solvers. This speed-up can help improve the quality of recommendations without affecting user experience, highlighting how optimization can provide solid orthogonal value to machine-learned recommender systems.