Goal-Directedness is in the Eye of the Beholder

TL;DR

This paper critically examines how goal-directedness in agents is perceived and measured, arguing it is subjective and proposing new perspectives on modeling it as an emergent property in multi-agent systems.

Contribution

It analyzes the assumptions behind behavioral and mechanistic approaches to goal attribution and suggests viewing goal-directedness as an emergent, system-level property.

Findings

Goal-directedness cannot be measured objectively.

Behavioral and mechanistic approaches have fundamental limitations.

Emergent properties offer a new modeling direction.

Abstract

Our ability to predict the behavior of complex agents turns on the attribution of goals. Probing for goal-directed behavior comes in two flavors: Behavioral and mechanistic. The former proposes that goal-directedness can be estimated through behavioral observation, whereas the latter attempts to probe for goals in internal model states. We work through the assumptions behind both approaches, identifying technical and conceptual problems that arise from formalizing goals in agent systems. We arrive at the perhaps surprising position that goal-directedness cannot be measured objectively. We outline new directions for modeling goal-directedness as an emergent property of dynamic, multi-agent systems.