A Machine with Short-Term, Episodic, and Semantic Memory Systems

TL;DR

This paper models an agent with short-term, episodic, and semantic memory systems inspired by human cognition, and evaluates it in a custom reinforcement learning environment called "the Room."

Contribution

It introduces a novel memory-structured agent modeled with knowledge graphs and demonstrates its advantages over traditional agents in a new environment.

Findings

The agent learns to decide which memories to forget or store.

Memory-structured agent outperforms non-memory agents in the environment.

The system effectively encodes, stores, and retrieves memories for task success.

Abstract

Inspired by the cognitive science theory of the explicit human memory systems, we have modeled an agent with short-term, episodic, and semantic memory systems, each of which is modeled with a knowledge graph. To evaluate this system and analyze the behavior of this agent, we designed and released our own reinforcement learning agent environment, "the Room", where an agent has to learn how to encode, store, and retrieve memories to maximize its return by answering questions. We show that our deep Q-learning based agent successfully learns whether a short-term memory should be forgotten, or rather be stored in the episodic or semantic memory systems. Our experiments indicate that an agent with human-like memory systems can outperform an agent without this memory structure in the environment.