IDD-AW: A Benchmark for Safe and Robust Segmentation of Drive Scenes in Unstructured Traffic and Adverse Weather

TL;DR

The paper introduces the IDD-AW dataset, a large-scale, challenging benchmark for safe and robust segmentation of driving scenes under adverse weather and unstructured traffic, including a new safety-aware evaluation metric.

Contribution

It provides a comprehensive dataset with paired NIR images, a hierarchical label set, and benchmarks state-of-the-art models with a novel safety-focused metric.

Findings

IDD-AW is highly challenging for current models.

The new Safe mIoU metric effectively penalizes unsafe predictions.

Benchmark results highlight the need for more robust segmentation methods.

Abstract

Large-scale deployment of fully autonomous vehicles requires a very high degree of robustness to unstructured traffic, and weather conditions, and should prevent unsafe mispredictions. While there are several datasets and benchmarks focusing on segmentation for drive scenes, they are not specifically focused on safety and robustness issues. We introduce the IDD-AW dataset, which provides 5000 pairs of high-quality images with pixel-level annotations, captured under rain, fog, low light, and snow in unstructured driving conditions. As compared to other adverse weather datasets, we provide i.) more annotated images, ii.) paired Near-Infrared (NIR) image for each frame, iii.) larger label set with a 4-level label hierarchy to capture unstructured traffic conditions. We benchmark state-of-the-art models for semantic segmentation in IDD-AW. We also propose a new metric called ''Safe mean Intersection over Union (Safe mIoU)'' for hierarchical datasets which penalizes dangerous mispredictions that are not captured in the traditional definition of mean Intersection over Union (mIoU). The results show that IDD-AW is one of the most challenging datasets to date for these tasks. The dataset and code will be available here: http://iddaw.github.io.