QianfanHuijin Technical Report: A Novel Multi-Stage Training Paradigm for Finance Industrial LLMs

TL;DR

This paper introduces QianfanHuijin, a financial domain LLM trained through a multi-stage paradigm involving continual pre-training and progressive fine-tuning, significantly improving financial reasoning and agentic capabilities for industrial applications.

Contribution

It proposes a generalizable multi-stage training framework for financial LLMs, combining knowledge consolidation with targeted reasoning and agentic training stages.

Findings

QianfanHuijin outperforms existing financial benchmarks.

Targeted RL stages improve reasoning and agentic skills.

Ablation studies confirm the effectiveness of each training stage.

Abstract

Domain-specific enhancement of Large Language Models (LLMs) within the financial context has long been a focal point of industrial application. While previous models such as BloombergGPT and Baichuan-Finance primarily focused on knowledge enhancement, the deepening complexity of financial services has driven a growing demand for models that possess not only domain knowledge but also robust financial reasoning and agentic capabilities. In this paper, we present QianfanHuijin, a financial domain LLM, and propose a generalizable multi-stage training paradigm for industrial model enhancement. Our approach begins with Continual Pre-training (CPT) on financial corpora to consolidate the knowledge base. This is followed by a fine-grained Post-training pipeline designed with increasing specificity: starting with Financial SFT, progressing to Finance Reasoning RL and Finance Agentic RL, and culminating in General RL aligned with real-world business scenarios. Empirical results demonstrate that QianfanHuijin achieves superior performance across various authoritative financial benchmarks. Furthermore, ablation studies confirm that the targeted Reasoning RL and Agentic RL stages yield significant gains in their respective capabilities. These findings validate our motivation and suggest that this fine-grained, progressive post-training methodology is poised to become a mainstream paradigm for various industrial-enhanced LLMs.