Temporally Consistent Unsupervised Segmentation for Mobile Robot Perception

TL;DR

This paper introduces Frontier-Seg, a method for unsupervised, temporally consistent terrain segmentation in mobile robot video streams, eliminating the need for labeled data and improving perception in unstructured environments.

Contribution

Frontier-Seg is the first approach to enforce temporal consistency in unsupervised segmentation for mobile robot perception using foundation model features.

Findings

Effective unsupervised segmentation across diverse datasets

Maintains temporal consistency in terrain boundary detection

Operates without human supervision

Abstract

Rapid progress in terrain-aware autonomous ground navigation has been driven by advances in supervised semantic segmentation. However, these methods rely on costly data collection and labor-intensive ground truth labeling to train deep models. Furthermore, autonomous systems are increasingly deployed in unrehearsed, unstructured environments where no labeled data exists and semantic categories may be ambiguous or domain-specific. Recent zero-shot approaches to unsupervised segmentation have shown promise in such settings but typically operate on individual frames, lacking temporal consistency-a critical property for robust perception in unstructured environments. To address this gap we introduce Frontier-Seg, a method for temporally consistent unsupervised segmentation of terrain from mobile robot video streams. Frontier-Seg clusters superpixel-level features extracted from foundation model backbones-specifically DINOv2-and enforces temporal consistency across frames to identify persistent terrain boundaries or frontiers without human supervision. We evaluate Frontier-Seg on a diverse set of benchmark datasets-including RUGD and RELLIS-3D-demonstrating its ability to perform unsupervised segmentation across unstructured off-road environments.