TopicProphet: Prophesies on Temporal Topic Trends and Stocks

TL;DR

TopicProphet is a novel framework that leverages historical sentiment and background analysis through topic modeling and temporal segmentation to improve stock trend predictions despite the inherent unpredictability of stock markets.

Contribution

It introduces a new approach combining topic modeling, temporal analysis, and breakpoint detection to identify optimal training periods for stock prediction models.

Findings

Outperforms state-of-the-art methods in forecasting financial percentage changes.

Effectively captures socioeconomic and political patterns relevant to stock trends.

Addresses data scarcity by selecting relevant historical periods for training.

Abstract

Stocks can't be predicted. Despite many hopes, this premise held itself true for many years due to the nature of quantitative stock data lacking causal logic along with rapid market changes hindering accumulation of significant data for training models. To undertake this matter, we propose a novel framework, TopicProphet, to analyze historical eras that share similar public sentiment trends and historical background. Our research deviates from previous studies that identified impacts of keywords and sentiments - we expand on that method by a sequence of topic modeling, temporal analysis, breakpoint detection and segment optimization to detect the optimal time period for training. This results in improving predictions by providing the model with nuanced patterns that occur from that era's socioeconomic and political status while also resolving the shortage of pertinent stock data to train on. Through extensive analysis, we conclude that TopicProphet produces improved outcomes compared to the state-of-the-art methods in capturing the optimal training data for forecasting financial percentage changes.