Small Language Model Can Self-correct

TL;DR

This paper introduces Intrinsic Self-Correction (ISC), a method enabling small language models to spontaneously correct their outputs, significantly improving accuracy in reasoning tasks through fine-tuning with a novel Partial Answer Masking technique.

Contribution

The paper proposes ISC and PAM, a new approach for small LMs to self-correct without external prompts, enhancing their output quality in reasoning tasks.

Findings

ISC improves output accuracy over baseline models.

Self-correction benefits are consistent across different model sizes.

Method is effective for commonsense and factual reasoning tasks.

Abstract

Generative Language Models (LMs) such as ChatGPT have exhibited remarkable performance across various downstream tasks. Nevertheless, one of their most prominent drawbacks is generating inaccurate or false information with a confident tone. Previous studies have devised sophisticated pipelines and prompts to induce large LMs to exhibit the capability for self-correction. However, large LMs are explicitly prompted to verify and modify its answers separately rather than completing all steps spontaneously like humans. Moreover, these complex prompts are extremely challenging for small LMs to follow. In this paper, we introduce the \underline{I}ntrinsic \underline{S}elf-\underline{C}orrection (ISC) in generative language models, aiming to correct the initial output of LMs in a self-triggered manner, even for those small LMs with 6 billion parameters. Specifically, we devise a pipeline for constructing self-correction data and propose Partial Answer Masking (PAM), aiming to endow the model with the capability for intrinsic self-correction through fine-tuning. We conduct experiments using LMs with parameters sizes ranging from 6 billion to 13 billion in two tasks, including commonsense reasoning and factual knowledge reasoning. Our experiments demonstrate that the outputs generated using ISC outperform those generated without self-correction. We believe that the output quality of even small LMs can be further improved by empowering them with the ability to intrinsic self-correct.