Mapping how LLMs debate societal issues when shadowing human personality traits, sociodemographics and social media behavior

TL;DR

This paper introduces Cognitive Digital Shadows, a large synthetic corpus of LLM responses on societal issues, enabling analysis of how prompts influence language, bias, and social sensitivity across personas and models.

Contribution

It presents a novel, extensive dataset and prompting framework for analyzing LLMs' discourse variations across social and psychological attributes.

Findings

CDS contains 190,000 records from 19 LLMs on 4 societal topics.

The dataset links sociodemographic attributes to language and stance.

Enables interactive analysis of emotional and semantic framing across models.

Abstract

Large Language Models (LLMs) can strongly shape social discourse, yet datasets investigating how LLM outputs vary across controlled social and contextual prompting remain sparse. Cognitive Digital Shadows (CDS) is a 190,000-record synthetic corpus supporting analyses of LLM-generated discourse. Each CDS record is generated by one of 19 LLMs, prompted to shadow either a human persona or an AI-assistant role. CDS contains LLM responses on 4 controversial societal topics: vaccines/healthcare, social media disinformation, the gender gap in science, and STEM stereotypes. Persona-conditioned records encode 17 sociodemographic and psychological attributes, providing data linking LLMs' prompts, language, stances and reasoning. Texts are validated for topic anchoring and can support emotional analyses via interpretable NLP (e.g. textual forma mentis networks). CDS is enriched by a pooling platform with user-friendly dashboards, enabling easy, interactive group-level comparisons of emotional and semantic framing across personas, topics and models. The CDS prompting framework supports future audits of LLMs' bias, social sensitivity and alignment.