Machine Psychology
Thilo Hagendorff, Ishita Dasgupta, Marcel Binz, Stephanie C.Y. Chan,, Andrew Lampinen, Jane X. Wang, Zeynep Akata, Eric Schulz

TL;DR
This paper advocates for applying psychological experimental methods to large language models to better understand their emergent behaviors and reasoning abilities beyond traditional benchmarks.
Contribution
It introduces the concept of 'machine psychology,' synthesizes existing approaches, and discusses future directions for studying LLMs through experimental psychology techniques.
Findings
Highlights the potential of psychological experiments for LLM analysis
Synthesizes best practices for machine psychology research
Identifies challenges and future directions in the field
Abstract
Large language models (LLMs) show increasingly advanced emergent capabilities and are being incorporated across various societal domains. Understanding their behavior and reasoning abilities therefore holds significant importance. We argue that a fruitful direction for research is engaging LLMs in behavioral experiments inspired by psychology that have traditionally been aimed at understanding human cognition and behavior. In this article, we highlight and summarize theoretical perspectives, experimental paradigms, and computational analysis techniques that this approach brings to the table. It paves the way for a "machine psychology" for generative artificial intelligence (AI) that goes beyond performance benchmarks and focuses instead on computational insights that move us toward a better understanding and discovery of emergent abilities and behavioral patterns in LLMs. We review…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling
MethodsTest
