Scale-Invariant Learning-to-Rank

TL;DR

This paper introduces a scale-invariant learning-to-rank framework that ensures consistent feature scaling in production, improving ranking reliability despite feature scale discrepancies between training and deployment.

Contribution

The paper proposes a novel deep and wide neural network architecture that guarantees scale-invariance in learning-to-rank models during training and prediction.

Findings

Framework outperforms non-scale-invariant models under feature scale perturbations

Maintains ranking performance despite train-test feature scale discrepancies

Effective in simulated real-world scenarios with feature scale issues

Abstract

At Expedia, learning-to-rank (LTR) models plays a key role on our website in sorting and presenting information more relevant to users, such as search filters, property rooms, amenities, and images. A major challenge in deploying these models is ensuring consistent feature scaling between training and production data, as discrepancies can lead to unreliable rankings when deployed. Normalization techniques like feature standardization and batch normalization could address these issues but are impractical in production due to latency impacts and the difficulty of distributed real-time inference. To address consistent feature scaling issue, we introduce a scale-invariant LTR framework which combines a deep and a wide neural network to mathematically guarantee scale-invariance in the model at both training and prediction time. We evaluate our framework in simulated real-world scenarios with injected feature scale issues by perturbing the test set at prediction time, and show that even with inconsistent train-test scaling, using framework achieves better performance than without.