Case-Based Reasoning Approach for Solving Financial Question Answering

TL;DR

This paper introduces a case-based reasoning approach for financial question answering, leveraging similar past cases to improve numerical reasoning and complex multi-step program solving in financial documents.

Contribution

It proposes a novel CBR method that retrieves relevant cases to enhance reasoning in financial QA, addressing limitations of existing program generation approaches.

Findings

Competitive performance on FinQA dataset

Improved handling of multi-step programs

Case repository expansion enhances reasoning capabilities

Abstract

Measuring a machine's understanding of human language often involves assessing its reasoning skills, i.e. logical process of deriving answers to questions. While recent language models have shown remarkable proficiency in text based tasks, their efficacy in complex reasoning problems involving heterogeneous information such as text, tables, and numbers remain uncertain. Addressing this gap, FinQA introduced a numerical reasoning dataset for financial documents and simultaneously proposed a program generation approach . Our investigation reveals that half of the errors (48%) stem from incorrect operations being generated. To address this issue, we propose a novel approach to tackle numerical reasoning problems using case based reasoning (CBR), an artificial intelligence paradigm that provides problem solving guidance by offering similar cases (i.e. similar questions and corresponding logical programs). Our model retrieves relevant cases to address a given question, and then generates an answer based on the retrieved cases and contextual information. Through experiments on the FinQA dataset, we demonstrate competitive performance of our approach and additionally show that by expanding case repository, we can help solving complex multi step programs which FinQA showed weakness of.