Projective simulation for artificial intelligence

TL;DR

This paper introduces a novel learning agent model that uses a simulation-based projection mechanism, enabling it to anticipate future scenarios before acting, with potential applications in embodied cognition and quantum computation.

Contribution

It presents a new model of artificial intelligence based on a dynamic network of episodic memory clips that facilitates future projection and decision-making.

Findings

The model allows for dynamic adaptation of the memory network.

It integrates simulation with reinforcement learning principles.

Potential extension to quantum computational frameworks.

Abstract

We propose a model of a learning agent whose interaction with the environment is governed by a simulation-based projection, which allows the agent to project itself into future situations before it takes real action. Projective simulation is based on a random walk through a network of clips, which are elementary patches of episodic memory. The network of clips changes dynamically, both due to new perceptual input and due to certain compositional principles of the simulation process. During simulation, the clips are screened for specific features which trigger factual action of the agent. The scheme is different from other, computational, notions of simulation, and it provides a new element in an embodied cognitive science approach to intelligent action and learning. Our model provides a natural route for generalization to quantum-mechanical operation and connects the fields of reinforcement learning and quantum computation.