Scaling GraphLLM with Bilevel-Optimized Sparse Querying

TL;DR

This paper introduces BOSQ, a framework that reduces the computational cost of using large language models for node-level tasks on text-attributed graphs by selectively querying explanations, achieving significant speedups without sacrificing accuracy.

Contribution

BOSQ is a novel adaptive sparse querying strategy that efficiently leverages LLM explanations for graph tasks, reducing costs while maintaining or improving performance.

Findings

BOSQ achieves orders of magnitude speedup over existing methods.

BOSQ maintains or improves task performance compared to full-query approaches.

Experiments on six real-world datasets validate BOSQ's efficiency and effectiveness.

Abstract

LLMs have recently shown strong potential in enhancing node-level tasks on text-attributed graphs (TAGs) by providing explanation features. However, their practical use is severely limited by the high computational and monetary cost of repeated LLM queries. To illustrate, naively generating explanations for all nodes on a medium-sized benchmark like Photo (48k nodes) using a representative method (e.g., TAPE) would consume days of processing time. In this paper, we propose Bilevel-Optimized Sparse Querying (BOSQ), a general framework that selectively leverages LLM-derived explanation features to enhance performance on node-level tasks on TAGs. We design an adaptive sparse querying strategy that selectively decides when to invoke LLMs, avoiding redundant or low-gain queries and significantly reducing computation overhead. Extensive experiments on six real-world TAG datasets involving two types of node-level tasks demonstrate that BOSQ achieves orders of magnitude speedups over existing GraphLLM methods while consistently delivering on-par or superior performance.