Mind the Gap: The Divergence Between Human and LLM-Generated Tasks

TL;DR

This paper compares human and LLM-generated task creation, revealing significant differences in psychological influences, social and physical content, and embodiment, emphasizing the need for more human-like motivation in AI.

Contribution

It demonstrates that LLMs fail to replicate human psychological and embodied task generation, highlighting a core gap in current AI models.

Findings

Humans are influenced by personal values and cognitive styles in task creation.

LLMs produce less social and physical tasks, focusing more on abstraction.

Despite perceiving LLM tasks as fun and novel, they lack human-like embodied goals.

Abstract

Humans constantly generate a diverse range of tasks guided by internal motivations. While generative agents powered by large language models (LLMs) aim to simulate this complex behavior, it remains uncertain whether they operate on similar cognitive principles. To address this, we conducted a task-generation experiment comparing human responses with those of an LLM agent (GPT-4o). We find that human task generation is consistently influenced by psychological drivers, including personal values (e.g., Openness to Change) and cognitive style. Even when these psychological drivers are explicitly provided to the LLM, it fails to reflect the corresponding behavioral patterns. They produce tasks that are markedly less social, less physical, and thematically biased toward abstraction. Interestingly, while the LLM's tasks were perceived as more fun and novel, this highlights a disconnect between its linguistic proficiency and its capacity to generate human-like, embodied goals. We conclude that there is a core gap between the value-driven, embodied nature of human cognition and the statistical patterns of LLMs, highlighting the necessity of incorporating intrinsic motivation and physical grounding into the design of more human-aligned agents.