Egocentric Spatial Memory

TL;DR

This paper introduces a deep neural network architecture for egocentric spatial memory that constructs global maps from egocentric views, enabling long-term place recognition and loop closure in large environments.

Contribution

The proposed ESM network models egocentric spatial memory without feature engineering, handling continuous actions and states, and integrates external memory for long-term place recognition.

Findings

Accurate global mapping in 3D virtual mazes

Robust performance in realistic indoor environments

Effective long-term place recognition

Abstract

Egocentric spatial memory (ESM) defines a memory system with encoding, storing, recognizing and recalling the spatial information about the environment from an egocentric perspective. We introduce an integrated deep neural network architecture for modeling ESM. It learns to estimate the occupancy state of the world and progressively construct top-down 2D global maps from egocentric views in a spatially extended environment. During the exploration, our proposed ESM model updates belief of the global map based on local observations using a recurrent neural network. It also augments the local mapping with a novel external memory to encode and store latent representations of the visited places over long-term exploration in large environments which enables agents to perform place recognition and hence, loop closure. Our proposed ESM network contributes in the following aspects: (1) without feature engineering, our model predicts free space based on egocentric views efficiently in an end-to-end manner; (2) different from other deep learning-based mapping system, ESMN deals with continuous actions and states which is vitally important for robotic control in real applications. In the experiments, we demonstrate its accurate and robust global mapping capacities in 3D virtual mazes and realistic indoor environments by comparing with several competitive baselines.