Empowering Small Language Models with Factual Hallucination-Aware Reasoning for Financial Classification

TL;DR

This paper introduces a three-step pipeline called AAAI that reduces factual hallucinations in small language models, thereby improving their accuracy in financial classification tasks.

Contribution

The paper presents a novel AAAI pipeline that detects and mitigates factual hallucinations in small language models to enhance financial classification performance.

Findings

Factual hallucinations correlate with misclassification.

Encoder-based verifiers effectively detect hallucinations.

Adaptive inference improves classification accuracy.

Abstract

Small language models (SLMs) are increasingly used for financial classification due to their fast inference and local deployability. However, compared with large language models, SLMs are more prone to factual hallucinations in reasoning and exhibit weaker classification performance. This raises a natural question: Can mitigating factual hallucinations improve SLMs' financial classification? To address this, we propose a three-step pipeline named AAAI (Association Identification, Automated Detection, and Adaptive Inference). Experiments on three representative SLMs reveal that: (1) factual hallucinations are positively correlated with misclassifications; (2) encoder-based verifiers effectively detect factual hallucinations; and (3) incorporating feedback on factual errors enables SLMs' adaptive inference that enhances classification performance. We hope this pipeline contributes to trustworthy and effective applications of SLMs in finance.