Multi-Modal Opinion Integration for Financial Sentiment Analysis using Cross-Modal Attention

TL;DR

This paper introduces a novel deep learning framework that effectively integrates recency and popularity opinion modalities using cross-modal attention for improved financial sentiment analysis, achieving high accuracy on a large dataset.

Contribution

It proposes a new cross-modal attention mechanism and an end-to-end model that combines textual opinion modalities for enhanced sentiment classification in finance.

Findings

Achieved 83.5% accuracy on a large dataset of 837 companies.

Significantly outperformed baseline models by 21%.

Demonstrated effectiveness of multi-modal integration for financial sentiment analysis.

Abstract

In recent years, financial sentiment analysis of public opinion has become increasingly important for market forecasting and risk assessment. However, existing methods often struggle to effectively integrate diverse opinion modalities and capture fine-grained interactions across them. This paper proposes an end-to-end deep learning framework that integrates two distinct modalities of financial opinions: recency modality (timely opinions) and popularity modality (trending opinions), through a novel cross-modal attention mechanism specifically designed for financial sentiment analysis. While both modalities consist of textual data, they represent fundamentally different information channels: recency-driven market updates versus popularity-driven collective sentiment. Our model first uses BERT (Chinese-wwm-ext) for feature embedding and then employs our proposed Financial Multi-Head Cross-Attention (FMHCA) structure to facilitate information exchange between these distinct opinion modalities. The processed features are optimized through a transformer layer and fused using multimodal factored bilinear pooling for classification into negative, neutral, and positive sentiment. Extensive experiments on a comprehensive dataset covering 837 companies demonstrate that our approach achieves an accuracy of 83.5%, significantly outperforming baselines including BERT+Transformer by 21 percent. These results highlight the potential of our framework to support more accurate financial decision-making and risk management.