What LLMs Think When You Don't Tell Them What to Think About?
Yongchan Kwon, James Zou

TL;DR
This study investigates the intrinsic generative behaviors of various large language models when given minimal, neutral prompts, revealing their topical preferences, content specialization, and unique degeneration patterns.
Contribution
It provides a comprehensive analysis of LLMs' unconstrained output behaviors, highlighting systematic topical biases and content characteristics across multiple models.
Findings
Models exhibit strong and systematic topical preferences.
Content depth and specialization vary significantly between models.
Degeneration patterns include repetitive phrases and model-specific behaviors.
Abstract
Characterizing the behavior of large language models (LLMs) across diverse settings is critical for reliable monitoring and AI safety. However, most existing analyses rely on topic- or task-specific prompts, which can substantially limit what can be observed. In this work, we study what LLMs generate from minimal, topic-neutral inputs and probe their near-unconstrained generative behavior. Despite the absence of explicit topics, model outputs cover a broad semantic space, and surprisingly, each model family exhibits strong and systematic topical preferences. GPT-OSS predominantly generates programming (27.1%) and mathematical content (24.6%), whereas Llama most frequently generates literary content (9.1%). DeepSeek often generates religious content, while Qwen frequently generates multiple-choice questions. Beyond topical preferences, we also observe differences in content…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsArtificial Intelligence in Healthcare and Education · Topic Modeling · Scientific Computing and Data Management
