Value-Action Alignment in Large Language Models under Privacy-Prosocial Conflict

TL;DR

This paper introduces a new assessment protocol and metric to evaluate how well large language models' expressed privacy and prosocial values predict their data-sharing behaviors, revealing significant variability across models.

Contribution

It develops a context-based evaluation method and the VAAR metric to analyze value-action alignment in LLMs under conflicting privacy and prosocial motivations.

Findings

Stable but model-specific privacy-prosocial profiles

Substantial heterogeneity in value-action alignment

Introduction of a new assessment protocol and VAAR metric

Abstract

Large language models (LLMs) are increasingly used to simulate decision-making tasks involving personal data sharing, where privacy concerns and prosocial motivations can push choices in opposite directions. Existing evaluations often measure privacy-related attitudes or sharing intentions in isolation, which makes it difficult to determine whether a model's expressed values jointly predict its downstream data-sharing actions as in real human behaviors. We introduce a context-based assessment protocol that sequentially administers standardized questionnaires for privacy attitudes, prosocialness, and acceptance of data sharing within a bounded, history-carrying session. To evaluate value-action alignments under competing attitudes, we use multi-group structural equation modeling (MGSEM) to identify relations from privacy concerns and prosocialness to data sharing. We propose Value-Action Alignment Rate (VAAR), a human-referenced directional agreement metric that aggregates path-level evidence for expected signs. Across multiple LLMs, we observe stable but model-specific Privacy-PSA-AoDS profiles, and substantial heterogeneity in value-action alignment.