LambdaRankIC: Directly Optimizing Rank IC for Financial Prediction

TL;DR

This paper introduces LambdaRankIC, a novel method that directly optimizes Rank IC for financial prediction models, leading to improved ranking performance and financial metrics.

Contribution

LambdaRankIC is a new learning-to-rank approach that directly optimizes Rank IC by deriving lambda gradients, implemented in XGBoost, and shown to outperform existing methods.

Findings

LambdaRankIC accurately recovers true ranking in noiseless simulations.

It outperforms regression and NDCG-based methods under noisy conditions.

It achieves the best out-of-sample performance on financial metrics like Rank IC and Sharpe ratio.

Abstract

In financial predictions, the performance of machine learning models is often assessed by Rank IC, which is the Spearman rank correlation between the model predictions and the realized asset returns. Despite its wide adoption, most existing models are trained using regression losses or ranking objectives that may not align with Rank IC. We propose LambdaRankIC, a novel learning-to-rank approach that directly optimizes Rank IC. We circumvent the non-differentiability of the ranking operator by deriving the closed-form expression for the lambda gradients induced by the pairwise rank swaps, which enables efficient gradient-based optimization within the LambdaRank framework. We implement LambdaRankIC as a custom objective in XGBoost. Theoretically, we show that our approach optimizes an upper bound on Rank IC. We evaluate the proposed approach on both simulated and real-world financial data. In simulation studies, LambdaRankIC accurately recovers the true ranking structure in noiseless settings and consistently outperforms regression-based and NDCG-oriented ranking methods under low signal-to-noise ratios and heavy-tailed noise regimes. In empirical experiments using real market data, LambdaRankIC achieves the best out-of-sample performance on evaluation metrics commonly used in finance, including Rank IC, ICIR, monthly return, and Sharpe ratio. These results show that directly optimizing Rank IC can yield substantial improvements over conventional learning objectives in financial predictions when the full-order ranking quality is the primary goal.