Unify Graph Learning with Text: Unleashing LLM Potentials for Session Search

TL;DR

This paper introduces a novel approach that combines graph structures and large language models to improve session search by converting session graphs into text and training LLMs with symbolic learning tasks.

Contribution

It proposes Symbolic Graph Ranker (SGR), integrating graph-based and text-based methods using grammar rules and self-supervised learning to enhance LLMs' understanding of graph structures in session search.

Findings

Outperforms existing methods on AOL and Tiangong-ST datasets.

Effectively captures graph topologies within textual inputs.

Demonstrates the potential of combining symbolic graph conversion with LLM training.

Abstract

Session search involves a series of interactive queries and actions to fulfill user's complex information need. Current strategies typically prioritize sequential modeling for deep semantic understanding, overlooking the graph structure in interactions. While some approaches focus on capturing structural information, they use a generalized representation for documents, neglecting the word-level semantic modeling. In this paper, we propose Symbolic Graph Ranker (SGR), which aims to take advantage of both text-based and graph-based approaches by leveraging the power of recent Large Language Models (LLMs). Concretely, we first introduce a set of symbolic grammar rules to convert session graph into text. This allows integrating session history, interaction process, and task instruction seamlessly as inputs for the LLM. Moreover, given the natural discrepancy between LLMs pre-trained on textual corpora, and the symbolic language we produce using our graph-to-text grammar, our objective is to enhance LLMs' ability to capture graph structures within a textual format. To achieve this, we introduce a set of self-supervised symbolic learning tasks including link prediction, node content generation, and generative contrastive learning, to enable LLMs to capture the topological information from coarse-grained to fine-grained. Experiment results and comprehensive analysis on two benchmark datasets, AOL and Tiangong-ST, confirm the superiority of our approach. Our paradigm also offers a novel and effective methodology that bridges the gap between traditional search strategies and modern LLMs.