Deep Episodic Memory: Encoding, Recalling, and Predicting Episodic Experiences for Robot Action Execution

TL;DR

This paper introduces a deep neural network for robot episodic memory that encodes, recalls, and predicts actions, enabling improved action understanding and prediction in robots through a novel unsupervised model.

Contribution

The paper presents a new unsupervised deep episodic memory architecture that encodes, recalls, and predicts robot actions, with demonstrated effectiveness on large datasets and real-world robot scenarios.

Findings

Conceptually similar actions are mapped into the same latent space region.

The model achieves effective action retrieval and prediction.

Successful generalization demonstrated on a humanoid robot.

Abstract

We present a novel deep neural network architecture for representing robot experiences in an episodic-like memory which facilitates encoding, recalling, and predicting action experiences. Our proposed unsupervised deep episodic memory model 1) encodes observed actions in a latent vector space and, based on this latent encoding, 2) infers most similar episodes previously experienced, 3) reconstructs original episodes, and 4) predicts future frames in an end-to-end fashion. Results show that conceptually similar actions are mapped into the same region of the latent vector space. Based on these results, we introduce an action matching and retrieval mechanism, benchmark its performance on two large-scale action datasets, 20BN-something-something and ActivityNet and evaluate its generalization capability in a real-world scenario on a humanoid robot.