Generalization Boundaries of Fine-Tuned Small Language Models for Graph Structural Inference

TL;DR

This paper investigates how small language models fine-tuned for graph property estimation perform beyond their training conditions, focusing on generalization across graph size and family, with empirical results on real-world benchmarks.

Contribution

It systematically analyzes the generalization boundaries of fine-tuned small language models on graph inference tasks across different graph sizes and types.

Findings

Models maintain ordinal consistency across graph families.

Performance persists on larger graphs than training data.

Degradation profiles vary by architecture.

Abstract

Small language models fine-tuned for graph property estimation have demonstrated strong in-distribution performance, yet their generalization capabilities beyond training conditions remain poorly understood. In this work, we systematically investigate the boundaries of structural inference in fine-tuned small language models along two generalization axes - graph size and graph family distribution - and assess domain-learning capability on real-world graph benchmarks. Using a controlled experimental setup with three instruction-tuned models in the 3-4B parameter class and two graph serialization formats, we evaluate performance on graphs substantially larger than the training range and across held-out random graph families. Our results show that fine-tuned models maintain strong ordinal consistency across structurally distinct graph families and continue to rank graphs by structural properties on inputs substantially larger than those seen during training, with distinct architecture-specific degradation profiles. These findings delineate where fine-tuned small language models generalize reliably, providing empirical grounding for their use in graph-based reasoning tasks.