Misconfidence-based Demonstration Selection for LLM In-Context Learning

TL;DR

This paper introduces In-Context Reflection (ICR), a novel demonstration selection method for large language models that uses misconfidence to iteratively improve in-context learning without external supervision.

Contribution

The paper proposes ICR, a new iterative demonstration selection technique based on misconfidence, reducing reliance on external supervision and improving LLM performance across diverse tasks.

Findings

ICR achieves an average of 4% performance boost over existing methods.

ICR demonstrates strong cross-task generalization.

The method effectively identifies challenging examples to refine demonstrations.

Abstract

In-context learning with large language models (LLMs) excels at adapting to various tasks rapidly. However, its success hinges on carefully selecting demonstrations, which remains an obstacle in practice. Current approaches to this problem either rely on hard-to-acquire external supervision or require frequent interactions with LLMs, resulting in high costs. We propose a new method called In-Context Reflection (ICR) to overcome these challenges. ICR strategically selects demonstrations to reduce the discrepancy between the LLM's outputs and the actual input-output mappings. Specifically, ICR starts with a random set of initial demonstrations, then iteratively refines it. In each step, it analyzes a pool of candidate examples and identifies the ones most likely to challenge the LLM's current understanding, measured by a new metric called misconfidence. These most confusing examples are then selected to replace the less informative demonstrations in the current set. Our comprehensive evaluation across five diverse datasets encompassing 13 subtasks shows the efficacy of ICR. Compared to existing methods, ICR achieves an average performance boost of 4%, while demonstrating remarkable cross-task generalization capabilities.