Goals and the Structure of Experience

TL;DR

This paper proposes a novel computational framework where goal-directed state representations in agents co-emerge from interactions, integrating descriptive and prescriptive aspects of world models based on Buddhist epistemology.

Contribution

It introduces a goal-driven, interdependent model of world states that unifies behavioral, phenomenological, and neural perspectives of purposeful behavior.

Findings

Defines telic states as classes of goal-equivalent experiences.

Shows how divergence between policies and desirable features guides learning.

Supports the framework with empirical and theoretical literature.

Abstract

Purposeful behavior is a hallmark of natural and artificial intelligence. Its acquisition is often believed to rely on world models, comprising both descriptive (what is) and prescriptive (what is desirable) aspects that identify and evaluate state of affairs in the world, respectively. Canonical computational accounts of purposeful behavior, such as reinforcement learning, posit distinct components of a world model comprising a state representation (descriptive aspect) and a reward function (prescriptive aspect). However, an alternative possibility, which has not yet been computationally formulated, is that these two aspects instead co-emerge interdependently from an agent's goal. Here, we describe a computational framework of goal-directed state representation in cognitive agents, in which the descriptive and prescriptive aspects of a world model co-emerge from agent-environment interaction sequences, or experiences. Drawing on Buddhist epistemology, we introduce a construct of goal-directed, or telic, states, defined as classes of goal-equivalent experience distributions. Telic states provide a parsimonious account of goal-directed learning in terms of the statistical divergence between behavioral policies and desirable experience features. We review empirical and theoretical literature supporting this novel perspective and discuss its potential to provide a unified account of behavioral, phenomenological and neural dimensions of purposeful behaviors across diverse substrates.