RLPO: Residual Listwise Preference Optimization for Long-Context Review Ranking

TL;DR

RLPO is a novel ranking method that combines pointwise scoring with lightweight listwise residual correction, improving long-context review ranking performance efficiently.

Contribution

The paper introduces RLPO, a new listwise ranking approach that enhances pointwise LLM scores with residuals, addressing computational and stability issues in long-context settings.

Findings

RLPO outperforms strong baselines in NDCG@k metrics.

RLPO remains stable and effective as candidate list length increases.

A new large-scale benchmark for long-context review ranking is proposed.

Abstract

Review ranking is pivotal in e-commerce for prioritizing diagnostic and authentic feedback from the deluge of user-generated content. While large language models have improved semantic assessment, existing ranking paradigms face a persistent trade-off in long-context settings. Pointwise scoring is efficient but often fails to account for list-level interactions, leading to miscalibrated top-$k$ rankings. Listwise approaches can leverage global context, yet they are computationally expensive and become unstable as candidate lists grow. To address this, we propose Residual Listwise Preference Optimization (RLPO), which formulates ranking as listwise representation-level residual correction over a strong pointwise LLM scorer. RLPO first produces calibrated pointwise scores and item representations, then applies a lightweight encoder over the representations to predict listwise score residuals, avoiding full token-level listwise processing. We also introduce a large-scale benchmark for long-context review ranking with human verification. Experiments show RLPO improves NDCG@k over strong pointwise and listwise baselines and remains robust as list length increases.