FastGAS: Fast Graph-based Annotation Selection for In-Context Learning

TL;DR

FastGAS is a graph-based method that efficiently selects diverse, representative instances for in-context learning, significantly reducing selection time while improving prompt quality across various tasks and model sizes.

Contribution

We introduce FastGAS, a novel graph partitioning approach for rapid, high-quality instance selection in ICL, outperforming prior methods in speed and effectiveness.

Findings

FastGAS reduces selection time compared to existing methods.

It achieves higher performance on multiple tasks.

Effective for larger language models.

Abstract

In-context learning (ICL) empowers large language models (LLMs) to tackle new tasks by using a series of training instances as prompts. Since generating the prompts needs to sample from a vast pool of instances and annotate them (e.g., add labels in classification task), existing methods have proposed to select a subset of unlabeled examples for annotation, thus enhancing the quality of prompts and concurrently mitigating annotation costs. However, these methods often require a long time to select instances due to their complexity, hindering their practical viability. To address this limitation, we propose a graph-based selection method, FastGAS, designed to efficiently identify high-quality instances while minimizing computational overhead. Initially, we construct a data similarity graph based on instance similarities. Subsequently, employing a graph partitioning algorithm, we partition the graph into pieces. Within each piece (i.e., subgraph), we adopt a greedy approach to pick the most representative nodes. By aggregating nodes from diverse pieces and annotating the corresponding instances, we identify a set of diverse and representative instances for ICL. Compared to prior approaches, our method not only exhibits superior performance on different tasks but also significantly reduces selection time. In addition, we demonstrate the efficacy of our approach in LLMs of larger sizes.