Towards Unified Multimodal Financial Forecasting: Integrating Sentiment Embeddings and Market Indicators via Cross-Modal Attention

TL;DR

This paper introduces STONK, a multimodal framework that combines numerical market data and sentiment-enriched news embeddings using cross-modal attention to enhance daily stock-movement prediction.

Contribution

It presents a novel unified approach integrating textual and numerical data with attention mechanisms for improved financial forecasting.

Findings

STONK outperforms numeric-only baselines in backtesting.

Fusion strategies significantly impact prediction accuracy.

The framework provides scalable solutions for multimodal financial analysis.

Abstract

We propose STONK (Stock Optimization using News Knowledge), a multimodal framework integrating numerical market indicators with sentiment-enriched news embeddings to improve daily stock-movement prediction. By combining numerical & textual embeddings via feature concatenation and cross-modal attention, our unified pipeline addresses limitations of isolated analyses. Backtesting shows STONK outperforms numeric-only baselines. A comprehensive evaluation of fusion strategies and model configurations offers evidence-based guidance for scalable multimodal financial forecasting. Source code is available on GitHub