Learning Explainable Stock Predictions with Tweets Using Mixture of Experts

TL;DR

This paper introduces FTS-Text-MoE, a novel model combining numerical stock data with textual information from news and social media using a Mixture of Experts Transformer, improving prediction accuracy and computational efficiency.

Contribution

The paper presents a new multi-modal model that integrates textual summaries with numerical data via MoE transformers, addressing input length and resource limitations in stock prediction.

Findings

Outperforms baseline models in investment returns

Achieves higher Sharpe ratios in experiments

Demonstrates effective multi-scale forecasting

Abstract

Stock price movements are influenced by many factors, and alongside historical price data, tex-tual information is a key source. Public news and social media offer valuable insights into market sentiment and emerging events. These sources are fast-paced, diverse, and significantly impact future stock trends. Recently, LLMs have enhanced financial analysis, but prompt-based methods still have limitations, such as input length restrictions and difficulties in predicting sequences of varying lengths. Additionally, most models rely on dense computational layers, which are resource-intensive. To address these challenges, we propose the FTS- Text-MoE model, which combines numerical data with key summaries from news and tweets using point embeddings, boosting prediction accuracy through the integration of factual textual data. The model uses a Mixture of Experts (MoE) Transformer decoder to process both data types. By activating only a subset of model parameters, it reduces computational costs. Furthermore, the model features multi-resolution prediction heads, enabling flexible forecasting of financial time series at different scales. Experimental results show that FTS-Text-MoE outperforms baseline methods in terms of investment returns and Sharpe ratio, demonstrating its superior accuracy and ability to predict future market trends.