Principles Do Not Apply Themselves: A Hermeneutic Perspective on AI Alignment

TL;DR

This paper argues that AI alignment involves interpretive judgments about principles, which are context-dependent and often only observable in deployment behavior, challenging purely principle-based approaches.

Contribution

It introduces a hermeneutic perspective to AI alignment, emphasizing the importance of interpretive, context-sensitive judgments in applying principles.

Findings

Preference-labeling data often involves conflicting or indifferent principles.

Alignment-relevant decisions are reflected in model responses at deployment.

Off-policy audits may miss alignment failures due to distribution differences.

Abstract

AI alignment is often framed as the task of ensuring that an AI system follows a set of stated principles or human preferences, but general principles rarely determine their own application in concrete cases. When principles conflict, when they are too broad to settle a situation, or when the relevant facts are unclear, an additional act of judgment is required. This paper analyzes that step through the lens of hermeneutics and argues that alignment therefore includes an interpretive component: it involves context-sensitive judgments about how principles should be read, applied, and prioritized in practice. We connect this claim to recent empirical findings showing that a substantial portion of preference-labeling data falls into cases of principle conflict or indifference, where the principle set does not uniquely determine a decision. We then draw an operational consequence: because such judgments are expressed in behavior, many alignment-relevant choices appear only in the distribution of responses a model generates at deployment time. To formalize this point, we distinguish deployment-induced and corpus-induced evaluation and show that off-policy audits can fail to capture alignment-relevant failures when the two response distributions differ. We argue that principle-specified alignment includes a context-dependent interpretive component.