LLM Cognitive Judgements Differ From Human
Sotiris Lamprinidis
TL;DR
This paper investigates GPT-3 and ChatGPT's performance on a cognitive science inductive reasoning task, revealing that their cognitive judgments significantly differ from human reasoning patterns.
Contribution
It provides an empirical comparison showing that LLMs' cognitive judgments are not aligned with human-like reasoning, highlighting limitations in their cognitive modeling.
Findings
LLMs' judgments differ from humans in inductive reasoning tasks
GPT-3 and ChatGPT show limited human-like cognitive capabilities
Results suggest LLMs may not serve as accurate models of human cognition
Abstract
Large Language Models (LLMs) have lately been on the spotlight of researchers, businesses, and consumers alike. While the linguistic capabilities of such models have been studied extensively, there is growing interest in investigating them as cognitive subjects. In the present work I examine GPT-3 and ChatGPT capabilities on an limited-data inductive reasoning task from the cognitive science literature. The results suggest that these models' cognitive judgements are not human-like.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Artificial Intelligence in Healthcare and Education · Natural Language Processing Techniques
Methods15 Ways to Contact How can i speak to someone at Delta Airlines · Attention Is All You Need · Adam · Layer Normalization · Attention Dropout · Linear Layer · Softmax · Cosine Annealing · Dense Connections · Weight Decay