Impact of LLMs news Sentiment Analysis on Stock Price Movement Prediction

TL;DR

This study evaluates the impact of different LLM-based news sentiment analysis models on stock price movement prediction, demonstrating that ensemble approaches and sentiment features can improve prediction accuracy.

Contribution

It provides a comprehensive comparison of three LLMs for sentiment analysis in stock prediction and assesses the benefit of sentiment features across various models.

Findings

DeBERTa achieves 75% accuracy, outperforming others.

Ensemble model improves accuracy to 80%.

Sentiment features slightly benefit certain prediction models.

Abstract

This paper addresses stock price movement prediction by leveraging LLM-based news sentiment analysis. Earlier works have largely focused on proposing and assessing sentiment analysis models and stock movement prediction methods, however, separately. Although promising results have been achieved, a clear and in-depth understanding of the benefit of the news sentiment to this task, as well as a comprehensive assessment of different architecture types in this context, is still lacking. Herein, we conduct an evaluation study that compares 3 different LLMs, namely, DeBERTa, RoBERTa and FinBERT, for sentiment-driven stock prediction. Our results suggest that DeBERTa outperforms the other two models with an accuracy of 75% and that an ensemble model that combines the three models can increase the accuracy to about 80%. Also, we see that sentiment news features can benefit (slightly) some stock market prediction models, i.e., LSTM-, PatchTST- and tPatchGNN-based classifiers and PatchTST- and TimesNet-based regression tasks models.