CFGPT: Chinese Financial Assistant with Large Language Model

TL;DR

This paper introduces CFGPT, a comprehensive Chinese financial language model framework that includes specialized datasets, a tailored LLM, and an application deployment system for real-world financial tasks.

Contribution

The work presents a new Chinese financial LLM framework with dedicated datasets, training procedures, and deployment tools, advancing AI capabilities in financial applications.

Findings

Developed CFData dataset with 584M documents and 141B tokens.

Trained CFLLM based on InternLM-7B for financial tasks.

Created CFAPP for deploying financial language models in real-world scenarios.

Abstract

Large language models (LLMs) have demonstrated great potential in natural language processing tasks within the financial domain. In this work, we present a Chinese Financial Generative Pre-trained Transformer framework, named CFGPT, which includes a dataset~(CFData) for pre-training and supervised fine-tuning, a financial LLM~(CFLLM) to adeptly manage financial texts, and a deployment framework~(CFAPP) designed to navigate real-world financial applications. The CFData comprising both a pre-training dataset and a supervised fine-tuning dataset, where the pre-training dataset collates Chinese financial data and analytics, alongside a smaller subset of general-purpose text with 584M documents and 141B tokens in total, and the supervised fine-tuning dataset is tailored for six distinct financial tasks, embodying various facets of financial analysis and decision-making with 1.5M instruction pairs and 1.5B tokens in total. The CFLLM, which is based on InternLM-7B to balance the model capability and size, is trained on CFData in two stage, continued pre-training and supervised fine-tuning. The CFAPP is centered on large language models (LLMs) and augmented with additional modules to ensure multifaceted functionality in real-world application. Our codes are released at https://github.com/TongjiFinLab/CFGPT.