Beyond Demonstrations: Dynamic Vector Construction from Latent Representations

TL;DR

This paper introduces DyVec, a dynamic vector construction method that improves task adaptation in large language models by robustly extracting and injecting semantically meaningful latent representations during inference.

Contribution

DyVec employs an exhaustive query rotation and dynamic segmentation to enhance latent representation robustness and optimizes injection positions, advancing inference-time adaptation techniques.

Findings

DyVec outperforms few-shot ICL, LoRA, and prior ICV methods.

Dynamic segmentation improves representation relevance.

Optimized injection positions enhance task-specific performance.

Abstract

In-Context derived Vector (ICV) methods extract task-relevant representations from large language models (LLMs) and reinject them during inference, achieving comparable performance to few-shot In-Context Learning (ICL) without repeated demonstration processing. However, existing ICV methods remain sensitive to ICL-specific factors, often use coarse or semantically fragmented representations as the source of the vector, and rely on heuristic-based injection positions, limiting their applicability. To address these issues, we propose Dynamic Vector (DyVec), which incorporates an Exhaustive Query Rotation (EQR) strategy to extract robust semantically aggregated latent representations by mitigating variance introduced by ICL. It then applies Dynamic Latent Segmentation and Injection to adaptively partition representations based on task complexity and leverages REINFORCE-based optimization to learn optimal injection positions for each segment. Experiments results show that DyVec outperforms few-shot ICL, LoRA, and prior ICV baselines. Further analysis highlights the effectiveness of dynamically segmenting and injecting semantically aggregated latent representations. DyVec provides a lightweight and data-efficient solution for inference-time task adaptation.