OT-Drive: Out-of-Distribution Off-Road Traversable Area Segmentation via Optimal Transport

TL;DR

OT-Drive introduces an optimal transport-based multi-modal fusion framework that significantly improves out-of-distribution traversable area segmentation in autonomous driving, enhancing robustness and generalization in unstructured environments.

Contribution

The paper presents a novel Scene Anchor Generator and OT Fusion module that leverage optimal transport for robust multi-modal scene understanding in OOD scenarios.

Findings

Achieves 95.16% mIoU on ORFD OOD scenarios, outperforming prior methods.

Surpasses baselines by 13.99% on cross-dataset transfer tasks.

Demonstrates strong OOD generalization with limited training data.

Abstract

Reliable traversable area segmentation in unstructured environments is critical for planning and decision-making in autonomous driving. However, existing data-driven approaches often suffer from degraded segmentation performance in out-of-distribution (OOD) scenarios, consequently impairing downstream driving tasks. To address this issue, we propose OT-Drive, an Optimal Transport--driven multi-modal fusion framework. The proposed method formulates RGB and surface normal fusion as a distribution transport problem. Specifically, we design a novel Scene Anchor Generator (SAG) to decompose scene information into the joint distribution of weather, time-of-day, and road type, thereby constructing semantic anchors that can generalize to unseen scenarios. Subsequently, we design an innovative Optimal Transport-based multi-modal fusion module (OT Fusion) to transport RGB and surface normal features onto the manifold defined by the semantic anchors, enabling robust traversable area segmentation under OOD scenarios. Experimental results demonstrate that our method achieves 95.16% mIoU on ORFD OOD scenarios, outperforming prior methods by 6.35%, and 89.79% mIoU on cross-dataset transfer tasks, surpassing baselines by 13.99%.These results indicate that the proposed model can attain strong OOD generalization with only limited training data, substantially enhancing its practicality and efficiency for real-world deployment.