LabelFusion: Fusing Large Language Models with Transformer Encoders for Robust Financial News Classification

TL;DR

This paper introduces LabelFusion, a hybrid model combining large language models and transformer encoders for robust financial news classification, demonstrating superior performance with ample data and competitive results in low-data scenarios.

Contribution

The paper proposes LabelFusion, a novel hybrid architecture that fuses LLM prompts with transformer embeddings, improving classification accuracy in financial news analysis.

Findings

LabelFusion achieves 96.0% macro F1 on full data.

LLMs perform well even in zero-shot settings.

LabelFusion outperforms standalone models with sufficient data.

Abstract

Financial news plays a central role in shaping investor sentiment and short-term dynamics in commodity markets. Many downstream financial applications, such as commodity price prediction or sentiment modeling, therefore rely on the ability to automatically identify news articles relevant to specific assets. However, obtaining large labeled corpora for financial text classification is costly, and transformer-based classifiers such as RoBERTa often degrade significantly in low-data regimes. Our results show that appropriately prompted out-of-the-box Large Language Models (LLMs) achieve strong performance even in such settings. Furthermore, we propose LabelFusion, a hybrid architecture that combines the output of a prompt-engineered LLM with contextual embeddings produced by a fine-tuned RoBERTa encoder through a lightweight Multilayer Perceptron (MLP) voting layer. Evaluated on a ten-class multi-label subset of the Reuters-21578 corpus, LabelFusion achieves a macro F1 score of 96.0% and an accuracy of 92.3% when trained on the full dataset, outperforming both standalone RoBERTa (F1 94.6%) and the standalone LLM (F1 93.9%). In low- to mid-data regimes, however, the LLM alone proves surprisingly competitive, achieving an F1 score of 75.9% even in a zero-shot setting and consistently outperforming LabelFusion until approximately 80% of the training data is available. These results suggest that LLM-only prompting is the preferred strategy under annotation constraints, whereas LabelFusion becomes the most effective solution once sufficient labeled data is available to train the encoder component. The code is available in an anonymized repository.