Low-latency Perception in Off-Road Dynamical Low Visibility Environments

TL;DR

This paper introduces a new dataset and a modular deep learning framework for semantic segmentation to enable low-latency perception in off-road environments under adverse conditions, supporting autonomous vehicle safety.

Contribution

It presents a novel dataset for off-road environments with adverse conditions, a configurable segmentation network framework, and an evaluation of real-time inference feasibility for autonomous off-road perception.

Findings

Deep learning can effectively segment off-road obstacles under adverse conditions.

The CMSNet framework allows flexible architecture testing for real-time applications.

Field tests demonstrate the system's practical viability in low-visibility scenarios.

Abstract

This work proposes a perception system for autonomous vehicles and advanced driver assistance specialized on unpaved roads and off-road environments. In this research, the authors have investigated the behavior of Deep Learning algorithms applied to semantic segmentation of off-road environments and unpaved roads under differents adverse conditions of visibility. Almost 12,000 images of different unpaved and off-road environments were collected and labeled. It was assembled an off-road proving ground exclusively for its development. The proposed dataset also contains many adverse situations such as rain, dust, and low light. To develop the system, we have used convolutional neural networks trained to segment obstacles and areas where the car can pass through. We developed a Configurable Modular Segmentation Network (CMSNet) framework to help create different architectures arrangements and test them on the proposed dataset. Besides, we also have ported some CMSNet configurations by removing and fusing many layers using TensorRT, C++, and CUDA to achieve embedded real-time inference and allow field tests. The main contributions of this work are: a new dataset for unpaved roads and off-roads environments containing many adverse conditions such as night, rain, and dust; a CMSNet framework; an investigation regarding the feasibility of applying deep learning to detect region where the vehicle can pass through when there is no clear boundary of the track; a study of how our proposed segmentation algorithms behave in different severity levels of visibility impairment; and an evaluation of field tests carried out with semantic segmentation architectures ported for real-time inference.