The Emergence of Abstract and Episodic Neurons in Episodic Meta-RL

TL;DR

This paper investigates how different neuron types emerge in an episodic meta-reinforcement learning agent, revealing distinct roles for shared knowledge and episode-specific information in working memory.

Contribution

It introduces a detailed analysis of neuron classes in episodic meta-RL, highlighting the emergence of abstract and episodic neurons in the agent's working memory.

Findings

Abstract neurons encode shared task knowledge.

Episodic neurons carry episode-specific information.

Reinstatement mechanism reveals neuron class emergence.

Abstract

In this work, we analyze the reinstatement mechanism introduced by Ritter et al. (2018) to reveal two classes of neurons that emerge in the agent's working memory (an epLSTM cell) when trained using episodic meta-RL on an episodic variant of the Harlow visual fixation task. Specifically, Abstract neurons encode knowledge shared across tasks, while Episodic neurons carry information relevant for a specific episode's task.