Curate-Train-Refine: A Closed-Loop Agentic Framework for Zero Shot Classification

TL;DR

This paper introduces Curate-Train-Refine, a closed-loop framework where LLMs generate and refine training data to improve lightweight classifiers for zero-shot tasks, reducing inference costs while maintaining high accuracy.

Contribution

The paper presents a novel iterative, agentic framework that leverages LLMs to curate and refine training data, enhancing classifier performance without large-model inference.

Findings

Outperforms standard zero-shot baselines across four benchmarks.

Enables efficient classification with reduced inference costs.

Demonstrates LLMs as effective data curators for downstream tasks.

Abstract

Large language models (LLMs) and high-capacity encoders have advanced zero and few-shot classification, but their inference cost and latency limit practical deployment. We propose training lightweight text classifiers using dynamically generated supervision from an LLM. Our method employs an iterative, agentic loop in which the LLM curates training data, analyzes model successes and failures, and synthesizes targeted examples to address observed errors. This closed-loop generation and evaluation process progressively improves data quality and adapts it to the downstream classifier and task. Across four widely used benchmarks, our approach consistently outperforms standard zero and few-shot baselines. These results indicate that LLMs can serve effectively as data curators, enabling accurate and efficient classification without the operational cost of large-model deployment.