Fine-tuning of lightweight large language models for sentiment classification on heterogeneous financial textual data

TL;DR

This paper evaluates lightweight open-source large language models for financial sentiment analysis across diverse datasets, showing they can perform competitively with limited data and resources, offering a cost-effective alternative to larger models.

Contribution

It demonstrates that lightweight open-source LLMs can effectively generalize financial sentiment analysis across heterogeneous data sources with minimal training data.

Findings

Qwen3 8B and Llama3 8B perform best in most scenarios.

Models achieve good results with only 5% of training data.

Lightweight LLMs are a cost-effective alternative for financial NLP.

Abstract

Large language models (LLMs) play an increasingly important role in financial markets analysis by capturing signals from complex and heterogeneous textual data sources, such as tweets, news articles, reports, and microblogs. However, their performance is dependent on large computational resources and proprietary datasets, which are costly, restricted, and therefore inaccessible to many researchers and practitioners. To reflect realistic situations we investigate the ability of lightweight open-source LLMs -- smaller and publicly available models designed to operate with limited computational resources -- to generalize sentiment understanding from financial datasets of varying sizes, sources, formats, and languages. We compare the benchmark finance natural language processing (NLP) model, FinBERT, and three open-source lightweight LLMs, DeepSeek-LLM 7B, Llama3 8B Instruct, and Qwen3 8B on five publicly available datasets: FinancialPhraseBank, Financial Question Answering, Gold News Sentiment, Twitter Sentiment and Chinese Finance Sentiment. We find that LLMs, specially Qwen3 8B and Llama3 8B, perform best in most scenarios, even from using only 5% of the available training data. These results hold in zero-shot and few-shot learning scenarios. Our findings indicate that lightweight, open-source large language models (LLMs) constitute a cost-effective option, as they can achieve competitive performance on heterogeneous textual data even when trained on only a limited subset of the extensive annotated corpora that are typically deemed necessary.