Deep Bayesian Future Fusion for Self-Supervised, High-Resolution, Off-Road Mapping

TL;DR

This paper introduces Future Fusion, a deep learning framework that generates dense, high-resolution off-road maps from sparse sensor data, improving navigation safety and accuracy in challenging environments.

Contribution

It presents a novel integration of Bayes filtering with deep learning and perceptual losses to produce high-resolution maps from sparse data, outperforming traditional methods.

Findings

Outperforms conventional baselines in dense map generation

Improves downstream navigation performance

Effectively handles sparse stereo and LiDAR data

Abstract

High-speed off-road navigation requires long-range, high-resolution maps to enable robots to safely navigate over different surfaces while avoiding dangerous obstacles. However, due to limited computational power and sensing noise, most approaches to off-road mapping focus on producing coarse (20-40cm) maps of the environment. In this paper, we propose Future Fusion, a framework capable of generating dense, high-resolution maps from sparse sensing data (30m forward at 2cm). This is accomplished by - (1) the efficient realization of the well-known Bayes filtering within the standard deep learning models that explicitly accounts for the sparsity pattern in stereo and LiDAR depth data, and (2) leveraging perceptual losses common in generative image completion. The proposed methodology outperforms the conventional baselines. Moreover, the learned features and the completed dense maps lead to improvements in the downstream navigation task.