Towards Non-Parametric Learning to Rank

TL;DR

This paper identifies flaws in existing nearest neighbor algorithms for learning-to-rank problems and proposes new algorithms utilizing global and local data information to improve neighbor identification in latent feature spaces.

Contribution

It reveals the incorrectness of Kendall-tau based kNN in ranking models and introduces novel algorithms leveraging global and local data features for better neighbor detection.

Findings

Kendall-tau based kNN produces incorrect results in the model.

New algorithms using global data features improve neighbor identification.

Different methods are used for finding similar agents and alternatives.

Abstract

This paper studies a stylized, yet natural, learning-to-rank problem and points out the critical incorrectness of a widely used nearest neighbor algorithm. We consider a model with $n$ agents (users) $\{x_i\}_{i \in [n]}$ and $m$ alternatives (items) $\{y_j\}_{j \in [m]}$, each of which is associated with a latent feature vector. Agents rank items nondeterministically according to the Plackett-Luce model, where the higher the utility of an item to the agent, the more likely this item will be ranked high by the agent. Our goal is to find neighbors of an arbitrary agent or alternative in the latent space. We first show that the Kendall-tau distance based kNN produces incorrect results in our model. Next, we fix the problem by introducing a new algorithm with features constructed from "global information" of the data matrix. Our approach is in sharp contrast to most existing feature engineering methods. Finally, we design another new algorithm identifying similar alternatives. The construction of alternative features can be done using "local information," highlighting the algorithmic difference between finding similar agents and similar alternatives.