Do Language Models Think Consistently? A Study of Value Preferences Across Varying Response Lengths

TL;DR

This study investigates whether large language models' value preferences are consistent across short and long responses, revealing weak correlations and highlighting the need for better methods to ensure consistent value expression.

Contribution

It provides the first systematic comparison of value preferences in LLMs between short-form and long-form responses across multiple models and settings.

Findings

Weak correlation between short and long-form value preferences

Weak correlation across different long-form generation settings

Argument specificity negatively impacts preference strength

Abstract

Evaluations of LLMs' ethical risks and value inclinations often rely on short-form surveys and psychometric tests, yet real-world use involves long-form, open-ended responses -- leaving value-related risks and preferences in practical settings largely underexplored. In this work, we ask: Do value preferences inferred from short-form tests align with those expressed in long-form outputs? To address this question, we compare value preferences elicited from short-form reactions and long-form responses, varying the number of arguments in the latter to capture users' differing verbosity preferences. Analyzing five LLMs (llama3-8b, gemma2-9b, mistral-7b, qwen2-7b, and olmo-7b), we find (1) a weak correlation between value preferences inferred from short-form and long-form responses across varying argument counts, and (2) similarly weak correlation between preferences derived from any two distinct long-form generation settings. (3) Alignment yields only modest gains in the consistency of value expression. Further, we examine how long-form generation attributes relate to value preferences, finding that argument specificity negatively correlates with preference strength, while representation across scenarios shows a positive correlation. Our findings underscore the need for more robust methods to ensure consistent value expression across diverse applications.