Financial Knowledge Large Language Model

TL;DR

This paper introduces a comprehensive framework for evaluating and enhancing financial knowledge in large language models, including a benchmark, a knowledge adaptation method, and a financial QA system.

Contribution

It presents IDEA-FinBench for financial knowledge assessment, IDEA-FinKER for domain adaptation, and IDEA-FinQA for financial question answering, advancing LLM applications in finance.

Findings

IDEA-FinBench effectively evaluates LLMs on financial exams.

IDEA-FinKER enables rapid adaptation of LLMs to financial knowledge.

IDEA-FinQA demonstrates improved financial question-answering performance.

Abstract

Artificial intelligence is making significant strides in the finance industry, revolutionizing how data is processed and interpreted. Among these technologies, large language models (LLMs) have demonstrated substantial potential to transform financial services by automating complex tasks, enhancing customer service, and providing detailed financial analysis. Firstly, we introduce IDEA-FinBench, an evaluation benchmark specifically tailored for assessing financial knowledge in large language models (LLMs). This benchmark utilizes questions from two globally respected and authoritative financial professional exams, aimimg to comprehensively evaluate the capability of LLMs to directly address exam questions pertinent to the finance sector. Secondly, we propose IDEA-FinKER, a Financial Knowledge Enhancement framework designed to facilitate the rapid adaptation of general LLMs to the financial domain, introducing a retrieval-based few-shot learning method for real-time context-level knowledge injection, and a set of high-quality financial knowledge instructions for fine-tuning any general LLM. Finally, we present IDEA-FinQA, a financial question-answering system powered by LLMs. This system is structured around a scheme of real-time knowledge injection and factual enhancement using external knowledge. IDEA-FinQA is comprised of three main modules: the data collector, the data querying module, and LLM-based agents tasked with specific functions.