Hardware-Aware Feature Extraction Quantisation for Real-Time Visual Odometry on FPGA Platforms

TL;DR

This paper presents a hardware-aware, quantised neural network architecture for real-time visual feature extraction on FPGA, enabling efficient visual odometry with high accuracy and performance in resource-constrained environments.

Contribution

It introduces an FPGA-optimized, quantised SuperPoint-based feature extraction model for visual odometry, combining model compression with hardware-aware design for real-time deployment.

Findings

Achieved 54 fps processing of 640x480 images on FPGA

Outperformed existing solutions in speed and efficiency

Quantisation techniques impact accuracy and performance

Abstract

Accurate position estimation is essential for modern navigation systems deployed in autonomous platforms, including ground vehicles, marine vessels, and aerial drones. In this context, Visual Simultaneous Localisation and Mapping (VSLAM) - which includes Visual Odometry - relies heavily on the reliable extraction of salient feature points from the visual input data. In this work, we propose an embedded implementation of an unsupervised architecture capable of detecting and describing feature points. It is based on a quantised SuperPoint convolutional neural network. Our objective is to minimise the computational demands of the model while preserving high detection quality, thus facilitating efficient deployment on platforms with limited resources, such as mobile or embedded systems. We implemented the solution on an FPGA System-on-Chip (SoC) platform, specifically the AMD/Xilinx Zynq UltraScale+, where we evaluated the performance of Deep Learning Processing Units (DPUs) and we also used the Brevitas library and the FINN framework to perform model quantisation and hardware-aware optimisation. This allowed us to process 640 x 480 pixel images at up to 54 fps on an FPGA platform, outperforming state-of-the-art solutions in the field. We conducted experiments on the TUM dataset to demonstrate and discuss the impact of different quantisation techniques on the accuracy and performance of the model in a visual odometry task.