Improving Financial Forecasting with a Synergistic LLM-Transformer Architecture: A Hybrid Approach to Stock Price Prediction

TL;DR

This paper introduces a hybrid LLM-Transformer model for stock price prediction that combines textual market sentiment with historical data, achieving improved accuracy and interpretability in financial forecasting.

Contribution

It presents a formalized framework integrating LLMs with Transformers for financial forecasting, addressing the interaction mechanisms between textual and numerical data.

Findings

Outperforms baseline with 5.28% RMSE reduction

Maintains robustness under noisy data

Enhances interpretability with confidence-weighted attention

Abstract

This study proposes a novel hybrid deep learning framework that integrates a Large Language Model (LLM) with a Transformer architecture for stock price forecasting. The research addresses a critical theoretical gap in existing approaches that empirically combine textual and numerical data without a formal understanding of their interaction mechanisms. We conceptualise a prompt-based LLM as a mathematically defined signal generator, capable of extracting directional market sentiment and an associated confidence score from financial news. These signals are then dynamically fused with structured historical price features through a noise-robust gating mechanism, enabling the Transformer to adaptively weigh semantic and quantitative information. Empirical evaluations demonstrate that the proposed Hybrid LLM-Transformer model significantly outperforms a Vanilla Transformer baseline, reducing the Root Mean Squared Error (RMSE) by 5.28% (p = 0.003). Moreover, ablation and robustness analyses confirm the model's stability under noisy conditions and its capacity to maintain interpretability through confidence-weighted attention. The findings provide both theoretical and empirical support for a paradigm shift from empirical observation to formalised modelling of LLM-Transformer interactions, paving the way toward explainable, noise-resilient, and semantically enriched financial forecasting systems.