Object-Centric Scene Representations using Active Inference

TL;DR

This paper introduces an active inference-based, object-centric generative model for scene understanding that enables robots to infer object categories and poses, outperforming traditional learning methods in a new viewpoint-matching benchmark.

Contribution

A novel hierarchical object-centric generative model combined with active inference for improved scene understanding and a new benchmark for evaluating active vision agents.

Findings

Agent balances epistemic and goal-driven behavior effectively.

Outperforms supervised and reinforcement learning baselines significantly.

Demonstrates robust inference of object categories and poses.

Abstract

Representing a scene and its constituent objects from raw sensory data is a core ability for enabling robots to interact with their environment. In this paper, we propose a novel approach for scene understanding, leveraging a hierarchical object-centric generative model that enables an agent to infer object category and pose in an allocentric reference frame using active inference, a neuro-inspired framework for action and perception. For evaluating the behavior of an active vision agent, we also propose a new benchmark where, given a target viewpoint of a particular object, the agent needs to find the best matching viewpoint given a workspace with randomly positioned objects in 3D. We demonstrate that our active inference agent is able to balance epistemic foraging and goal-driven behavior, and outperforms both supervised and reinforcement learning baselines by a large margin.