Auto-ICL: In-Context Learning without Human Supervision

TL;DR

Auto-ICL introduces a framework where large language models autonomously generate their own contexts, including examples and instructions, to improve in-context learning performance without human supervision, outperforming existing methods.

Contribution

The paper presents a novel automatic in-context learning framework enabling models to generate effective contexts independently, reducing reliance on human-provided data.

Findings

Model-generated contexts outperform human-annotated contexts.

Auto-ICL surpasses existing self-generated context methods.

Experiments show improved performance across models and datasets.

Abstract

With in-context learning ability, the performance of large language models can be significantly boosted when provided with appropriate context. However, existing in-context learning methods mainly rely on human-provided contexts, such as labeled examples and explicit instructions. Writing context by humans is labor-intensive on various tasks and limits the model to tasks manageable by humans. To overcome these limitations, we propose Automatic In-Context Learning framework that enables the model to autonomously generate examples and instructions for problem-solving. With experiments across various models and datasets, results show that model-generated contexts outperform human-annotated contexts, including Few-Shot and Few-Shot-CoT methods, and surpass existing self-generated context methods like Zero-CoT and Auto-CoT.