Relational Probing: LM-to-Graph Adaptation for Financial Prediction

TL;DR

This paper introduces Relational Probing, a method that directly induces relational graphs from language model hidden states for financial prediction, improving performance without increasing inference costs.

Contribution

It proposes a novel relational probing technique that replaces standard LM heads, enabling joint training for graph-based financial prediction tasks.

Findings

Relational Probing improves stock-trend prediction accuracy.

The method maintains low inference costs compared to traditional prompting.

Experiments with Qwen3 models demonstrate consistent performance gains.

Abstract

Language models can be used to identify relationships between financial entities in text. However, while structured output mechanisms exist, prompting-based pipelines still incur autoregressive decoding costs and decouple graph construction from downstream optimization. We propose \emph{Relational Probing}, which replaces the standard language-model head with a relation head that induces a relational graph directly from language-model hidden states and is trained jointly with the downstream task model for stock-trend prediction. This approach both learns semantic representations and preserves the strict structure of the induced relational graph. It enables language-model outputs to go beyond text, allowing them to be reshaped into task-specific formats for downstream models. To enhance reproducibility, we provide an operational definition of small language models (SLMs): models that can be fine-tuned end-to-end on a single 24GB GPU under specified batch-size and sequence-length settings. Experiments use Qwen3 backbones (0.6B/1.7B/4B) as upstream SLMs and compare against a co-occurrence baseline. Relational Probing yields consistent performance improvements at competitive inference cost.