Learning-on-the-Drive: Self-supervised Adaptation of Visual Offroad Traversability Models

TL;DR

This paper presents ALTER, a self-supervised learning framework that combines LiDAR and visual data to improve long-range offroad traversability prediction for autonomous vehicles, enabling better adaptation and safety.

Contribution

ALTER introduces a novel self-supervised visual adaptation method and a sensor failure response module for enhanced offroad navigation in unseen environments.

Findings

43.4% better traversability prediction than LiDAR-only methods

164% improvement over non-adaptive visual methods

Effective online learning within 45 seconds

Abstract

Autonomous offroad driving is essential for applications like emergency rescue, military operations, and agriculture. Despite progress, systems struggle with high-speed vehicles exceeding 10m/s due to the need for accurate long-range (> 50m) perception for safe navigation. Current approaches are limited by sensor constraints; LiDAR-based methods offer precise short-range data but are noisy beyond 30m, while visual models provide dense long-range measurements but falter with unseen scenarios. To overcome these issues, we introduce ALTER, a learning-on-the-drive perception framework that leverages both sensor types. ALTER uses a self-supervised visual model to learn and adapt from near-range LiDAR measurements, improving long-range prediction in new environments without manual labeling. It also includes a model selection module for better sensor failure response and adaptability to known environments. Testing in two real-world settings showed on average 43.4% better traversability prediction than LiDAR-only and 164% over non-adaptive state-of-the-art (SOTA) visual semantic methods after 45 seconds of online learning.