Can We Trust LLMs? Mitigate Overconfidence Bias in LLMs through Knowledge Transfer

TL;DR

This paper presents a knowledge transfer approach using chain of thoughts to reduce overconfidence in LLMs, resulting in more reliable and calibrated predictions across diverse tasks.

Contribution

It introduces a novel knowledge transfer method leveraging chain of thoughts to improve LLM calibration and accuracy, outperforming existing fine-tuning techniques.

Findings

KT method outperforms vanilla fine-tuning by 55.3%

KT improves confidence calibration across datasets

Enhanced trustworthiness and accuracy in LLM predictions

Abstract

The study explores mitigating overconfidence bias in LLMs to improve their reliability. We introduce a knowledge transfer (KT) method utilizing chain of thoughts, where "big" LLMs impart knowledge to "small" LLMs via detailed, sequential reasoning paths. This method uses advanced reasoning of larger models to fine-tune smaller models, enabling them to produce more accurate predictions with calibrated confidence. Experimental evaluation using multiple-choice questions and sentiment analysis across diverse datasets demonstrated the KT method's superiority over the vanilla and question-answer pair (QA) fine-tuning methods. The most significant improvement in three key metrics, where the KT method outperformed the vanilla and QA methods by an average of 55.3% and 43.1%, respectively. These findings underscore the KT method's potential in enhancing model trustworthiness and accuracy, offering precise outputs with well-matched confidence levels across various contexts.