AlignSurvey: A Comprehensive Benchmark for Human Preferences Alignment in Social Surveys
Chenxi Lin, Weikang Yuan, Zhuoren Jiang, Biao Huang, Ruitao Zhang, Jianan Ge, Yueqian Xu, Jianxing Yu

TL;DR
AlignSurvey introduces a comprehensive benchmark and dataset suite for evaluating large language models across the entire social survey process, emphasizing alignment, fairness, and cross-cultural validity.
Contribution
It presents the first full pipeline benchmark for social surveys using LLMs, including datasets, evaluation metrics, and reference models to improve alignment and fairness.
Findings
Effective evaluation metrics for survey response fidelity and fairness.
A large, cross-cultural dataset with 44K+ dialogues and 400K+ survey records.
Open-source models and tools for transparent social survey simulation.
Abstract
Understanding human attitudes, preferences, and behaviors through social surveys is essential for academic research and policymaking. Yet traditional surveys face persistent challenges, including fixed-question formats, high costs, limited adaptability, and difficulties ensuring cross-cultural equivalence. While recent studies explore large language models (LLMs) to simulate survey responses, most are limited to structured questions, overlook the entire survey process, and risks under-representing marginalized groups due to training data biases. We introduce AlignSurvey, the first benchmark that systematically replicates and evaluates the full social survey pipeline using LLMs. It defines four tasks aligned with key survey stages: social role modeling, semi-structured interview modeling, attitude stance modeling and survey response modeling. It also provides task-specific evaluation…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsSurvey Methodology and Nonresponse · Computational and Text Analysis Methods · Human Mobility and Location-Based Analysis
