Active Inference in Hebbian Learning Networks

TL;DR

This paper presents a brain-inspired active inference approach using Hebbian learning networks that learn environment dynamics and outperform traditional Q-learning in control tasks without replay buffers.

Contribution

It introduces a novel Hebbian learning-based active inference framework with separate ensembles for inference and prediction, demonstrating improved control performance.

Findings

Hebbian AIF outperforms Q-learning in Mountain Car environment

The approach learns environment dynamics without replay buffers

Parameter effects on task performance are analyzed

Abstract

This work studies how brain-inspired neural ensembles equipped with local Hebbian plasticity can perform active inference (AIF) in order to control dynamical agents. A generative model capturing the environment dynamics is learned by a network composed of two distinct Hebbian ensembles: a posterior network, which infers latent states given the observations, and a state transition network, which predicts the next expected latent state given current state-action pairs. Experimental studies are conducted using the Mountain Car environment from the OpenAI gym suite, to study the effect of the various Hebbian network parameters on the task performance. It is shown that the proposed Hebbian AIF approach outperforms the use of Q-learning, while not requiring any replay buffer, as in typical reinforcement learning systems. These results motivate further investigations of Hebbian learning for the design of AIF networks that can learn environment dynamics without the need for revisiting past buffered experiences.