LLM4PR: Improving Post-Ranking in Search Engine with Large Language Models

TL;DR

This paper introduces LLM4PR, a novel framework that uses large language models to improve post-ranking in search engines, leading to significant performance gains.

Contribution

The paper proposes a new paradigm with a Query-Instructed Adapter and feature adaptation to enhance post-ranking using LLMs, which is a largely unexplored area.

Findings

Achieves state-of-the-art post-ranking performance

Significant improvements over existing methods

Effective integration of LLMs in IR post-processing

Abstract

Alongside the rapid development of Large Language Models (LLMs), there has been a notable increase in efforts to integrate LLM techniques in information retrieval (IR) and search engines (SE). Recently, an additional post-ranking stage is suggested in SE to enhance user satisfaction in practical applications. Nevertheless, research dedicated to enhancing the post-ranking stage through LLMs remains largely unexplored. In this study, we introduce a novel paradigm named Large Language Models for Post-Ranking in search engine (LLM4PR), which leverages the capabilities of LLMs to accomplish the post-ranking task in SE. Concretely, a Query-Instructed Adapter (QIA) module is designed to derive the user/item representation vectors by incorporating their heterogeneous features. A feature adaptation step is further introduced to align the semantics of user/item representations with the LLM. Finally, the LLM4PR integrates a learning to post-rank step, leveraging both a main task and an auxiliary task to fine-tune the model to adapt the post-ranking task. Experiment studies demonstrate that the proposed framework leads to significant improvements and exhibits state-of-the-art performance compared with other alternatives.