Incorporating Pre-trained Model Prompting in Multimodal Stock Volume Movement Prediction

TL;DR

This paper introduces ProMUSE, a multimodal stock volume prediction model that leverages pre-trained language models and prompt learning to incorporate universal financial knowledge, improving prediction accuracy over existing methods.

Contribution

The paper proposes a novel multimodal prediction framework using prompt-based pre-trained models and a cross-modality contrastive alignment to enhance financial news understanding.

Findings

ProMUSE outperforms existing baselines in stock volume prediction.

The architecture effectively leverages pre-trained language models for financial news.

Cross-modality contrastive alignment improves multimodal fusion quality.

Abstract

Multimodal stock trading volume movement prediction with stock-related news is one of the fundamental problems in the financial area. Existing multimodal works that train models from scratch face the problem of lacking universal knowledge when modeling financial news. In addition, the models ability may be limited by the lack of domain-related knowledge due to insufficient data in the datasets. To handle this issue, we propose the Prompt-based MUltimodal Stock volumE prediction model (ProMUSE) to process text and time series modalities. We use pre-trained language models for better comprehension of financial news and adopt prompt learning methods to leverage their capability in universal knowledge to model textual information. Besides, simply fusing two modalities can cause harm to the unimodal representations. Thus, we propose a novel cross-modality contrastive alignment while reserving the unimodal heads beside the fusion head to mitigate this problem. Extensive experiments demonstrate that our proposed ProMUSE outperforms existing baselines. Comprehensive analyses further validate the effectiveness of our architecture compared to potential variants and learning mechanisms.