Object-based active inference

TL;DR

This paper introduces Object-based Active Inference (OBAI), a novel approach combining active inference with deep object-based neural networks to improve object segmentation and manipulation in AI agents.

Contribution

It presents a new model that integrates object-based representations into active inference, enabling better perception and control of objects in complex environments.

Findings

OBAI successfully segments objects in video input.

OBAI can manipulate objects towards arbitrary goals.

The model learns from experience in a simple environment.

Abstract

The world consists of objects: distinct entities possessing independent properties and dynamics. For agents to interact with the world intelligently, they must translate sensory inputs into the bound-together features that describe each object. These object-based representations form a natural basis for planning behavior. Active inference (AIF) is an influential unifying account of perception and action, but existing AIF models have not leveraged this important inductive bias. To remedy this, we introduce 'object-based active inference' (OBAI), marrying AIF with recent deep object-based neural networks. OBAI represents distinct objects with separate variational beliefs, and uses selective attention to route inputs to their corresponding object slots. Object representations are endowed with independent action-based dynamics. The dynamics and generative model are learned from experience with a simple environment (active multi-dSprites). We show that OBAI learns to correctly segment the action-perturbed objects from video input, and to manipulate these objects towards arbitrary goals.