Evaluating Financial Relational Graphs: Interpretation Before Prediction

TL;DR

This paper introduces a new dataset and evaluation methods for dynamic financial relationship graphs, enabling better interpretability and validation of graphs used in stock trend prediction before applying neural networks.

Contribution

The paper presents the SPNews dataset and novel evaluation techniques that assess financial relationship graphs independently of downstream prediction models.

Findings

Evaluation methods effectively differentiate between various financial graphs.

The proposed approach yields more interpretable financial relationship graphs.

Experimental results validate the effectiveness of the new evaluation techniques.

Abstract

Accurate and robust stock trend forecasting has been a crucial and challenging task, as stock price changes are influenced by multiple factors. Graph neural network-based methods have recently achieved remarkable success in this domain by constructing stock relationship graphs that reflect internal factors and relationships between stocks. However, most of these methods rely on predefined factors to construct static stock relationship graphs due to the lack of suitable datasets, failing to capture the dynamic changes in stock relationships. Moreover, the evaluation of relationship graphs in these methods is often tied to the performance of neural network models on downstream tasks, leading to confusion and imprecision. To address these issues, we introduce the SPNews dataset, collected based on S\&P 500 Index stocks, to facilitate the construction of dynamic relationship graphs. Furthermore, we propose a novel set of financial relationship graph evaluation methods that are independent of downstream tasks. By using the relationship graph to explain historical financial phenomena, we assess its validity before constructing a graph neural network, ensuring the graph's effectiveness in capturing relevant financial relationships. Experimental results demonstrate that our evaluation methods can effectively differentiate between various financial relationship graphs, yielding more interpretable results compared to traditional approaches. We make our source code publicly available on GitHub to promote reproducibility and further research in this area.