NDCG-Consistent Softmax Approximation with Accelerated Convergence

TL;DR

This paper introduces novel ranking loss functions derived from Softmax that improve computational efficiency and convergence speed in large-scale ranking tasks, while maintaining or enhancing ranking performance.

Contribution

It proposes the RG$^2$ and RG$^ imes$ losses based on Taylor expansions of Softmax, connecting weighted squared losses with ranking methods and integrating them with efficient ALS optimization.

Findings

Achieves comparable or better ranking performance than Softmax Loss.

Significantly accelerates convergence in large-scale ranking tasks.

Provides theoretical guarantees and convergence analysis for the proposed methods.

Abstract

Ranking tasks constitute fundamental components of extreme similarity learning frameworks, where extremely large corpora of objects are modeled through relative similarity relationships adhering to predefined ordinal structures. Among various ranking surrogates, Softmax (SM) Loss has been widely adopted due to its natural capability to handle listwise ranking via global negative comparisons, along with its flexibility across diverse application scenarios. However, despite its effectiveness, SM Loss often suffers from significant computational overhead and scalability limitations when applied to large-scale object spaces. To address this challenge, we propose novel loss formulations that align directly with ranking metrics: the Ranking-Generalizable \textbf{squared} (RG$^2$) Loss and the Ranking-Generalizable interactive (RG$^\times$) Loss, both derived through Taylor expansions of the SM Loss. Notably, RG$^2$ reveals the intrinsic mechanisms underlying weighted squared losses (WSL) in ranking methods and uncovers fundamental connections between sampling-based and non-sampling-based loss paradigms. Furthermore, we integrate the proposed RG losses with the highly efficient Alternating Least Squares (ALS) optimization method, providing both generalization guarantees and convergence rate analyses. Empirical evaluations on real-world datasets demonstrate that our approach achieves comparable or superior ranking performance relative to SM Loss, while significantly accelerating convergence. This framework offers the similarity learning community both theoretical insights and practically efficient tools, with methodologies applicable to a broad range of tasks where balancing ranking quality and computational efficiency is essential.