A Hierarchical Language Model For Interpretable Graph Reasoning

TL;DR

This paper introduces HLM-G, a hierarchical language model that improves graph reasoning by capturing local and global structures, enhancing interpretability, efficiency, and robustness in large-scale graph tasks.

Contribution

The paper presents a novel two-block hierarchical architecture for LLMs that significantly advances graph understanding and interpretability in large-scale graph reasoning tasks.

Findings

Outperforms existing methods in diverse graph reasoning tasks

Provides interpretable attention-based explanations

Reduces computational costs for large graphs

Abstract

Large language models (LLMs) are being increasingly explored for graph tasks. Despite their remarkable success in text-based tasks, LLMs' capabilities in understanding explicit graph structures remain limited, particularly with large graphs. In this work, we introduce Hierarchical Language Model for Graph (HLM-G), which employs a two-block architecture to capture node-centric local information and interaction-centric global structure, effectively enhancing graph structure understanding abilities. The proposed scheme allows LLMs to address various graph queries with high efficacy, efficiency, and robustness, while reducing computational costs on large-scale graph tasks. Furthermore, we demonstrate the interpretability of our model using intrinsic attention weights and established explainers. Comprehensive evaluations across diverse graph reasoning and real-world tasks of node, link, and graph-levels highlight the superiority of our method, marking a significant advancement in the application of LLMs to graph understanding.