SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation

TL;DR

SHIFT is a large synthetic dataset designed for autonomous driving that includes continuous environmental variations to facilitate research on domain adaptation and perception robustness.

Contribution

It introduces the first multi-task synthetic dataset with continuous domain shifts for autonomous driving, enabling evaluation of perception system robustness.

Findings

Supports investigation of perception degradation under domain shifts

Provides comprehensive annotations for multiple perception tasks

Facilitates development of continuous adaptation strategies

Abstract

Adapting to a continuously evolving environment is a safety-critical challenge inevitably faced by all autonomous driving systems. Existing image and video driving datasets, however, fall short of capturing the mutable nature of the real world. In this paper, we introduce the largest multi-task synthetic dataset for autonomous driving, SHIFT. It presents discrete and continuous shifts in cloudiness, rain and fog intensity, time of day, and vehicle and pedestrian density. Featuring a comprehensive sensor suite and annotations for several mainstream perception tasks, SHIFT allows investigating the degradation of a perception system performance at increasing levels of domain shift, fostering the development of continuous adaptation strategies to mitigate this problem and assess model robustness and generality. Our dataset and benchmark toolkit are publicly available at www.vis.xyz/shift.