How Reliable are LLMs for Reasoning on the Re-ranking task?

TL;DR

This paper investigates the reliability of Large Language Models in re-ranking tasks, analyzing how different training methods influence their semantic understanding and explainability, especially with limited data.

Contribution

It provides an in-depth analysis of how training methods impact LLMs' reasoning and explainability in re-ranking, highlighting the gap between semantic understanding and evaluation optimization.

Findings

Some training methods yield better explainability.

LLMs often gain abstract knowledge rather than true semantic understanding.

Explainability varies significantly across training approaches.

Abstract

With the improving semantic understanding capability of Large Language Models (LLMs), they exhibit a greater awareness and alignment with human values, but this comes at the cost of transparency. Although promising results are achieved via experimental analysis, an in-depth understanding of the LLM's internal workings is unavoidable to comprehend the reasoning behind the re-ranking, which provides end users with an explanation that enables them to make an informed decision. Moreover, in newly developed systems with limited user engagement and insufficient ranking data, accurately re-ranking content remains a significant challenge. While various training methods affect the training of LLMs and generate inference, our analysis has found that some training methods exhibit better explainability than others, implying that an accurate semantic understanding has not been learned through all training methods; instead, abstract knowledge has been gained to optimize evaluation, which raises questions about the true reliability of LLMs. Therefore, in this work, we analyze how different training methods affect the semantic understanding of the re-ranking task in LLMs and investigate whether these models can generate more informed textual reasoning to overcome the challenges of transparency or LLMs and limited training data. To analyze the LLMs for re-ranking tasks, we utilize a relatively small ranking dataset from the environment and the Earth science domain to re-rank retrieved content. Furthermore, we also analyze the explainable information to see if the re-ranking can be reasoned using explainability.