Self Iterative Label Refinement via Robust Unlabeled Learning

TL;DR

This paper introduces an iterative label refinement method using unlabeled data to improve large language model classification, reducing bias and enhancing performance across diverse tasks with minimal supervision.

Contribution

It proposes a novel unlabeled-unlabeled learning framework for iterative pseudo-label refinement, addressing biases in self-refinement of LLMs with minimal human supervision.

Findings

Outperforms initial LLM classification results.

Effective across diverse datasets including low-resource languages.

Enhances safety alignment and generative task performance.

Abstract

Recent advances in large language models (LLMs) have yielded impressive performance on various tasks, yet they often depend on high-quality feedback that can be costly. Self-refinement methods attempt to leverage LLMs' internal evaluation mechanisms with minimal human supervision; however, these approaches frequently suffer from inherent biases and overconfidence, especially in domains where the models lack sufficient internal knowledge, resulting in performance degradation. As an initial step toward enhancing self-refinement for broader applications, we introduce an iterative refinement pipeline that employs the Unlabeled-Unlabeled learning framework to improve LLM-generated pseudo-labels for classification tasks. By exploiting two unlabeled datasets with differing positive class ratios, our approach iteratively denoises and refines the initial pseudo-labels, thereby mitigating the adverse effects of internal biases with minimal human supervision. Evaluations on diverse datasets, including low-resource language corpora, patent classifications, and protein structure categorizations, demonstrate that our method consistently outperforms both initial LLM's classification performance and the self-refinement approaches by cutting-edge models (e.g., GPT-4o and DeepSeek-R1). Moreover, we experimentally confirm that our refined classifier facilitates effective post-training alignment for safety in LLMs and demonstrate successful self-refinement in generative tasks as well.\footnote{Our code is available at https://github.com/HikaruAsano/self-iterative-label-refinement.}