Affinity and Diversity: A Unified Metric for Demonstration Selection via Internal Representations

TL;DR

This paper introduces a unified metric based on internal representations, combining affinity and diversity, to improve demonstration selection in In-Context Learning, leading to more consistent and effective performance.

Contribution

It proposes a novel unified metric leveraging internal representations for demonstration selection, unifying previous inconsistent approaches.

Findings

Affinity and diversity strongly correlate with test accuracy

The proposed metrics unify various previous methods

Demonstration selection improves ICL performance

Abstract

The performance of In-Context Learning (ICL) is highly sensitive to the selected demonstrations. Existing approaches to demonstration selection optimize different objectives, yielding inconsistent results. To address this, we propose a unified metric--affinity and diversity--that leverages ICL model's internal representations. Our experiments show that both affinity and diversity strongly correlate with test accuracies, indicating their effectiveness for demonstration selection. Moreover, we show that our proposed metrics align well with various previous works to unify the inconsistency.