Do Prompts Reshape Representations? An Empirical Study of Prompting Effects on Embeddings

TL;DR

This study investigates how prompting influences the internal representations of language models, revealing that prompt relevance does not always correlate with improved embeddings, challenging common assumptions about prompt design.

Contribution

The paper provides an empirical analysis of prompt effects on embeddings, showing that prompt relevance does not consistently enhance representation quality.

Findings

Prompting affects representation quality but not always positively.

Relevance of prompts does not necessarily improve embeddings.

Factors influencing embedding changes are complex and not solely based on prompt relevance.

Abstract

Prompting is a common approach for leveraging LMs in zero-shot settings. However, the underlying mechanisms that enable LMs to perform diverse tasks without task-specific supervision remain poorly understood. Studying the relationship between prompting and the quality of internal representations can shed light on how pre-trained embeddings may support in-context task solving. In this empirical study, we conduct a series of probing experiments on prompt embeddings, analyzing various combinations of prompt templates for zero-shot classification. Our findings show that while prompting affects the quality of representations, these changes do not consistently correlate with the relevance of the prompts to the target task. This result challenges the assumption that more relevant prompts necessarily lead to better representations. We further analyze potential factors that may contribute to this unexpected behavior.