MARTI-4: new model of human brain, considering neocortex and basal ganglia -- learns to play Atari game by reinforcement learning on a single CPU

TL;DR

This paper introduces MARTI-4, a novel brain-inspired model combining neocortex and basal ganglia, which learns to play Atari games efficiently on a single CPU using reinforcement learning and a new surprise mechanism.

Contribution

It presents a new ML architecture called Deep Control and a brain-inspired model MARTI-4 that improves reinforcement learning with a surprise-based inner reward system.

Findings

MARTI-4 learns Atari games on a single CPU in hours

The surprise mechanism enhances reinforcement learning efficiency

Deep Control architecture uses cortical columns as structural units

Abstract

We present Deep Control - new ML architecture of cortico-striatal brain circuits, which use whole cortical column as a structural element, instead of a singe neuron. Based on this architecture, we present MARTI - new model of human brain, considering neocortex and basal ganglia. This model is de-signed to implement expedient behavior and is capable to learn and achieve goals in unknown environments. We introduce a novel surprise feeling mechanism, that significantly improves reinforcement learning process through inner rewards. We use OpenAI Gym environment to demonstrate MARTI learning on a single CPU just in several hours.