Hybrid guided variational autoencoder for visual place recognition

TL;DR

This paper introduces a hybrid guided variational autoencoder leveraging event-based vision and spiking neural networks to improve visual place recognition for mobile robots, achieving robustness, efficiency, and high generalization in indoor environments.

Contribution

It presents a novel guided VAE model compatible with neuromorphic hardware, combining event-based sensors and deep learning for robust, compact visual place recognition.

Findings

Achieves state-of-the-art classification performance on indoor VPR dataset.

Demonstrates robustness under various illumination conditions.

Shows high generalization to unseen scenes.

Abstract

Autonomous agents such as cars, robots and drones need to precisely localize themselves in diverse environments, including in GPS-denied indoor environments. One approach for precise localization is visual place recognition (VPR), which estimates the place of an image based on previously seen places. State-of-the-art VPR models require high amounts of memory, making them unwieldy for mobile deployment, while more compact models lack robustness and generalization capabilities. This work overcomes these limitations for robotics using a combination of event-based vision sensors and an event-based novel guided variational autoencoder (VAE). The encoder part of our model is based on a spiking neural network model which is compatible with power-efficient low latency neuromorphic hardware. The VAE successfully disentangles the visual features of 16 distinct places in our new indoor VPR dataset with a classification performance comparable to other state-of-the-art approaches while, showing robust performance also under various illumination conditions. When tested with novel visual inputs from unknown scenes, our model can distinguish between these places, which demonstrates a high generalization capability by learning the essential features of location. Our compact and robust guided VAE with generalization capabilities poses a promising model for visual place recognition that can significantly enhance mobile robot navigation in known and unknown indoor environments.