Is Interpretable Machine Learning Effective at Feature Selection for Neural Learning-to-Rank?

TL;DR

This paper evaluates the effectiveness of interpretable machine learning methods for feature selection in neural learning-to-rank models, revealing significant feature redundancy and potential efficiency gains.

Contribution

It compares six feature selection methods, introduces a new global method G-L2X, and demonstrates their impact on neural ranking model efficiency and interpretability.

Findings

Local method TabNet achieves optimal performance with fewer than 10 features.

Global methods like G-L2X improve efficiency with slightly more features.

Significant feature redundancy exists in LTR benchmarks.

Abstract

Neural ranking models have become increasingly popular for real-world search and recommendation systems in recent years. Unlike their tree-based counterparts, neural models are much less interpretable. That is, it is very difficult to understand their inner workings and answer questions like how do they make their ranking decisions? or what document features do they find important? This is particularly disadvantageous since interpretability is highly important for real-world systems. In this work, we explore feature selection for neural learning-to-rank (LTR). In particular, we investigate six widely-used methods from the field of interpretable machine learning (ML) and introduce our own modification, to select the input features that are most important to the ranking behavior. To understand whether these methods are useful for practitioners, we further study whether they contribute to efficiency enhancement. Our experimental results reveal a large feature redundancy in several LTR benchmarks: the local selection method TabNet can achieve optimal ranking performance with less than 10 features; the global methods, particularly our G-L2X, require slightly more selected features, but exhibit higher potential in improving efficiency. We hope that our analysis of these feature selection methods will bring the fields of interpretable ML and LTR closer together.