Navigation-Oriented Scene Understanding for Robotic Autonomy: Learning to Segment Driveability in Egocentric Images

TL;DR

This paper introduces a navigation-oriented image segmentation approach that classifies scenes into driveability levels, enhancing robotic navigation by focusing on safety-critical areas and improving generalization across diverse outdoor environments.

Contribution

It proposes a novel affordance-based segmentation method with soft ordinal labels and a pixel-wise loss weighting tailored for autonomous navigation tasks.

Findings

Improves driveability segmentation accuracy over standard methods.

Enhances cross-dataset generalization for outdoor scene understanding.

Prioritizes safety-critical areas in segmentation to support autonomous navigation.

Abstract

This work tackles scene understanding for outdoor robotic navigation, solely relying on images captured by an on-board camera. Conventional visual scene understanding interprets the environment based on specific descriptive categories. However, such a representation is not directly interpretable for decision-making and constrains robot operation to a specific domain. Thus, we propose to segment egocentric images directly in terms of how a robot can navigate in them, and tailor the learning problem to an autonomous navigation task. Building around an image segmentation network, we present a generic affordance consisting of 3 driveability levels which can broadly apply to both urban and off-road scenes. By encoding these levels with soft ordinal labels, we incorporate inter-class distances during learning which improves segmentation compared to standard "hard" one-hot labelling. In addition, we propose a navigation-oriented pixel-wise loss weighting method which assigns higher importance to safety-critical areas. We evaluate our approach on large-scale public image segmentation datasets ranging from sunny city streets to snowy forest trails. In a cross-dataset generalization experiment, we show that our affordance learning scheme can be applied across a diverse mix of datasets and improves driveability estimation in unseen environments compared to general-purpose, single-dataset segmentation.