Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learning

TL;DR

This paper introduces Bidirectional Alignment (BiAlign), a novel method that improves in-context learning in smaller models by aligning both input preferences and output distributions with larger teacher models, leading to better performance.

Contribution

The paper proposes a new alignment technique that considers input preferences alongside output distributions, enhancing the transfer of ICL capabilities from teacher to student models.

Findings

BiAlign outperforms existing methods across multiple tasks.

Incorporating input preference alignment improves ICL performance.

Extensive experiments validate the effectiveness of BiAlign.

Abstract

Large language models (LLMs) have shown impressive few-shot generalization on many tasks via in-context learning (ICL). Despite their success in showing such emergent abilities, the scale and complexity of larger models also lead to unprecedentedly high computational demands and deployment challenges. In reaction, researchers explore transferring the powerful capabilities of larger models to more efficient and compact models by typically aligning the output of smaller (student) models with that of larger (teacher) models. Existing methods either train student models on the generated outputs of teacher models or imitate their token-level probability distributions. However, these distillation methods pay little to no attention to the input, which also plays a crucial role in ICL. Based on the finding that the performance of ICL is highly sensitive to the selection of demonstration examples, we propose Bidirectional Alignment (BiAlign) to fully leverage the models' preferences for ICL examples to improve the ICL abilities of student models. Specifically, we introduce the alignment of input preferences between student and teacher models by incorporating a novel ranking loss, in addition to aligning the token-level output distribution. With extensive experiments and analysis, we demonstrate that BiAlign can consistently outperform existing baselines on a variety of tasks involving language understanding, reasoning, and coding.