A Numerical Reasoning Question Answering System with Fine-grained Retriever and the Ensemble of Multiple Generators for FinQA

TL;DR

This paper presents a numerical reasoning question answering system for financial data, combining a novel cell retriever, multiple program generators, and an ensemble module, achieving high accuracy in financial QA tasks.

Contribution

It introduces a cell retriever for precise data selection and an ensemble of multiple generators to improve numerical reasoning in financial QA.

Findings

Achieved 69.79% execution accuracy on FinQA private test set.

Designed a cell retriever to avoid irrelevant data in question answering.

Utilized multiple generators and ensemble methods for improved reasoning.

Abstract

The numerical reasoning in the financial domain -- performing quantitative analysis and summarizing the information from financial reports -- can greatly increase business efficiency and reduce costs of billions of dollars. Here, we propose a numerical reasoning question answering system to answer numerical reasoning questions among financial text and table data sources, consisting of a retriever module, a generator module, and an ensemble module. Specifically, in the retriever module, in addition to retrieving the whole row data, we innovatively design a cell retriever that retrieves the gold cells to avoid bringing unrelated and similar cells in the same row to the inputs of the generator module. In the generator module, we utilize multiple generators to produce programs, which are operation steps to answer the question. Finally, in the ensemble module, we integrate multiple programs to choose the best program as the output of our system. In the final private test set in FinQA Competition, our system obtains 69.79 execution accuracy.