Organization of a Latent Space structure in VAE/GAN trained by navigation data

TL;DR

This paper introduces a VAE/GAN-based cognitive mapping system that self-organizes its latent space to reflect dataset proximity, enabling internal generation of temporal sequences akin to hippocampal replay with added novelty.

Contribution

It demonstrates how VAE/GAN can organize a latent space reflecting data structure and generate temporal sequences with stability and novelty, mimicking cognitive processes.

Findings

Latent space distances reflect image similarities.

VAE/GAN generates stable and novel temporal sequences.

Self-organization of latent space supports cognitive mapping.

Abstract

We present a novel artificial cognitive mapping system using generative deep neural networks, called variational autoencoder/generative adversarial network (VAE/GAN), which can map input images to latent vectors and generate temporal sequences internally. The results show that the distance of the predicted image is reflected in the distance of the corresponding latent vector after training. This indicates that the latent space is self-organized to reflect the proximity structure of the dataset and may provide a mechanism through which many aspects of cognition are spatially represented. The present study allows the network to internally generate temporal sequences that are analogous to the hippocampal replay/pre-play ability, where VAE produces only near-accurate replays of past experiences, but by introducing GANs, the generated sequences are coupled with instability and novelty.