Enabling and Analyzing How to Efficiently Extract Information from Hybrid Long Documents with LLMs

TL;DR

This paper introduces the AFIE framework that significantly improves LLMs' ability to extract numerical information from complex hybrid financial reports, demonstrating substantial accuracy gains on GPT-3.5 and GPT-4.

Contribution

The paper presents a novel AFIE framework and a new FINE dataset to enhance and evaluate LLMs' performance in extracting information from hybrid long financial documents.

Findings

Average accuracy increase of 53.94% with GPT-3.5

Average accuracy increase of 33.77% with GPT-4

Effective validation on hybrid long documents

Abstract

Large Language Models (LLMs) demonstrate exceptional performance in textual understanding and tabular reasoning tasks. However, their ability to comprehend and analyze hybrid text, containing textual and tabular data, remains underexplored. In this research, we specialize in harnessing the potential of LLMs to comprehend critical information from financial reports, which are hybrid long-documents. We propose an Automated Financial Information Extraction (AFIE) framework that enhances LLMs' ability to comprehend and extract information from financial reports. To evaluate AFIE, we develop a Financial Reports Numerical Extraction (FINE) dataset and conduct an extensive experimental analysis. Our framework is effectively validated on GPT-3.5 and GPT-4, yielding average accuracy increases of 53.94% and 33.77%, respectively, compared to a naive method. These results suggest that the AFIE framework offers accuracy for automated numerical extraction from complex, hybrid documents.