ZiGong 1.0: A Large Language Model for Financial Credit

TL;DR

ZiGong 1.0 is a specialized large language model tailored for financial credit assessment, employing multi-task fine-tuning and a novel data pruning method to reduce hallucinations and improve accuracy in financial tasks.

Contribution

The paper introduces ZiGong, a financial domain-specific LLM with a new data pruning technique to mitigate hallucinations and enhance robustness in financial applications.

Findings

Significant reduction in hallucinations during financial tasks.

Improved prediction accuracy in real-world financial scenarios.

Enhanced model robustness through data refinement.

Abstract

Large Language Models (LLMs) have demonstrated strong performance across various general Natural Language Processing (NLP) tasks. However, their effectiveness in financial credit assessment applications remains suboptimal, primarily due to the specialized financial expertise required for these tasks. To address this limitation, we propose ZiGong, a Mistral-based model enhanced through multi-task supervised fine-tuning. To specifically combat model hallucination in financial contexts, we introduce a novel data pruning methodology. Our approach utilizes a proxy model to score training samples, subsequently combining filtered data with original datasets for model training. This data refinement strategy effectively reduces hallucinations in LLMs while maintaining reliability in downstream financial applications. Experimental results show our method significantly enhances model robustness and prediction accuracy in real-world financial scenarios.