Dial-insight: Fine-tuning Large Language Models with High-Quality Domain-Specific Data Preventing Capability Collapse

TL;DR

This paper presents a two-stage method for fine-tuning large language models with high-quality, domain-specific data, improving domain proficiency without losing generalization, validated on real estate interaction data.

Contribution

It introduces a novel prompt construction and quality assessment framework for domain-specific fine-tuning of LLMs, preserving their general capabilities.

Findings

Enhanced domain-specific performance of LLMs

Maintained generalization abilities after fine-tuning

Cost-effective data quality assurance framework

Abstract

The efficacy of large language models (LLMs) is heavily dependent on the quality of the underlying data, particularly within specialized domains. A common challenge when fine-tuning LLMs for domain-specific applications is the potential degradation of the model's generalization capabilities. To address these issues, we propose a two-stage approach for the construction of production prompts designed to yield high-quality data. This method involves the generation of a diverse array of prompts that encompass a broad spectrum of tasks and exhibit a rich variety of expressions. Furthermore, we introduce a cost-effective, multi-dimensional quality assessment framework to ensure the integrity of the generated labeling data. Utilizing a dataset comprised of service provider and customer interactions from the real estate sector, we demonstrate a positive correlation between data quality and model performance. Notably, our findings indicate that the domain-specific proficiency of general LLMs can be enhanced through fine-tuning with data produced via our proposed method, without compromising their overall generalization abilities, even when exclusively domain-specific data is employed for fine-tuning.