Real-World Robot Control by Deep Active Inference With a Temporally Hierarchical World Model

TL;DR

This paper introduces a deep active inference framework with a hierarchical world model for real-world robot control, enabling goal-directed and exploratory actions under uncertainty with efficient computation.

Contribution

It proposes a novel hierarchical deep active inference approach combining multiple timescale dynamics and action abstraction, improving real-world robot manipulation performance.

Findings

Achieves high success rates in diverse manipulation tasks

Effectively switches between goal-directed and exploratory actions

Reduces computational cost of action selection

Abstract

Robots in uncertain real-world environments must perform both goal-directed and exploratory actions. However, most deep learning-based control methods neglect exploration and struggle under uncertainty. To address this, we adopt deep active inference, a framework that accounts for human goal-directed and exploratory actions. Yet, conventional deep active inference approaches face challenges due to limited environmental representation capacity and high computational cost in action selection. We propose a novel deep active inference framework that consists of a world model, an action model, and an abstract world model. The world model encodes environmental dynamics into hidden state representations at slow and fast timescales. The action model compresses action sequences into abstract actions using vector quantization, and the abstract world model predicts future slow states conditioned on the abstract action, enabling low-cost action selection. We evaluate the framework on object-manipulation tasks with a real-world robot. Results show that it achieves high success rates across diverse manipulation tasks and switches between goal-directed and exploratory actions in uncertain settings, while making action selection computationally tractable. These findings highlight the importance of modeling multiple timescale dynamics and abstracting actions and state transitions.