# STARE at the Structure: Steering ICL Exemplar Selection with Structural Alignment

## TL;DR

This paper introduces a structure-aware exemplar selection method for in-context learning in semantic parsing, improving performance by aligning exemplars structurally and semantically, with minimal overhead.

## Contribution

It proposes a novel two-stage exemplar selection strategy that incorporates structural alignment via a fine-tuned retriever and a plug-in module, enhancing ICL effectiveness.

## Key findings

- Outperforms existing baselines on four semantic parsing benchmarks.
- Demonstrates consistent improvements across multiple LLMs.
- Efficient and easily integrable into existing ICL pipelines.

## Abstract

In-Context Learning (ICL) has become a powerful paradigm that enables LLMs to perform a wide range of tasks without task-specific fine-tuning. However, the effectiveness of ICL heavily depends on the quality of exemplar selection. In particular, for structured prediction tasks such as semantic parsing, existing ICL selection strategies often overlook structural alignment, leading to suboptimal performance and poor generalization. To address this issue, we propose a novel two-stage exemplar selection strategy that achieves a strong balance between efficiency, generalizability, and performance. First, we fine-tune a BERT-based retriever using structure-aware supervision, guiding it to select exemplars that are both semantically relevant and structurally aligned. Then, we enhance the retriever with a plug-in module, which amplifies syntactically meaningful information in the hidden representations. This plug-in is model-agnostic, requires minimal overhead, and can be seamlessly integrated into existing pipelines. Experiments on four benchmarks spanning three semantic parsing tasks demonstrate that our method consistently outperforms existing baselines with multiple recent LLMs as inference-time models.