CorrectionLM: Self-Corrections with SLM for Dialogue State Tracking

TL;DR

CorrectionLM introduces a self-correction framework for small language models in dialogue state tracking, enabling them to improve performance without relying on large models or extensive computation, especially in low-resource settings.

Contribution

It presents a novel correction method allowing SLMs to self-improve using in-context exemplars, reducing dependence on large models and computational costs.

Findings

Achieves state-of-the-art results comparable to large language models.

Reduces computational costs significantly compared to LLM-based correction methods.

Effective in low-resource dialogue state tracking tasks.

Abstract

Large language models (LLMs) have demonstrated self-improvement capabilities via feedback and refinement, but current small language models (SLMs) have had limited success in this area. Existing correction approaches often rely on distilling knowledge from LLMs, which imposes significant computation demands. In this work, we introduce CORRECTIONLM, a novel correction framework that enables SLMs to self-correct using in-context exemplars without LLM involvement. Applied to two dialogue state tracking (DST) tasks in low-resource settings, CORRECTIONLM achieves results similar to a state-of-the-art LLM at a small fraction of the computation costs.