Discovering material information using hierarchical Reformer model on financial regulatory filings

TL;DR

This paper introduces a hierarchical Reformer model that processes large financial regulatory documents to predict trade volume changes, revealing that regulatory filings contain material information relevant for market understanding.

Contribution

The paper presents a novel hierarchical Reformer model tailored for large document processing in finance, demonstrating its ability to predict market movements from regulatory filings.

Findings

Model successfully predicts trade volume changes from filings.

Attention patterns reveal detection of material information.

Processing regulatory filings benefits market understanding.

Abstract

Most applications of machine learning for finance are related to forecasting tasks for investment decisions. Instead, we aim to promote a better understanding of financial markets with machine learning techniques. Leveraging the tremendous progress in deep learning models for natural language processing, we construct a hierarchical Reformer ([15]) model capable of processing a large document level dataset, SEDAR, from canadian financial regulatory filings. Using this model, we show that it is possible to predict trade volume changes using regulatory filings. We adapt the pretraining task of HiBERT ([36]) to obtain good sentence level representations using a large unlabelled document dataset. Finetuning the model to successfully predict trade volume changes indicates that the model captures a view from financial markets and processing regulatory filings is beneficial. Analyzing the attention patterns of our model reveals that it is able to detect some indications of material information without explicit training, which is highly relevant for investors and also for the market surveillance mandate of financial regulators.