SUSTechGAN: Image Generation for Object Detection in Adverse Conditions of Autonomous Driving

TL;DR

This paper introduces SUSTechGAN, a novel GAN framework with attention modules and multi-scale generators, designed to generate adverse-condition driving images that enhance object detection performance in autonomous driving.

Contribution

SUSTechGAN's customized architecture and loss function effectively generate realistic adverse-condition images, improving object detection retraining in autonomous driving scenarios.

Findings

Generated images significantly improved YOLOv5 detection in rain and night.

SUSTechGAN outperforms existing GANs in adverse condition image generation.

Open-source code and datasets are provided for further research.

Abstract

Autonomous driving significantly benefits from data-driven deep neural networks. However, the data in autonomous driving typically fits the long-tailed distribution, in which the critical driving data in adverse conditions is hard to collect. Although generative adversarial networks (GANs) have been applied to augment data for autonomous driving, generating driving images in adverse conditions is still challenging. In this work, we propose a novel framework, SUSTechGAN, with customized dual attention modules, multi-scale generators, and a novel loss function to generate driving images for improving object detection of autonomous driving in adverse conditions. We test the SUSTechGAN and the well-known GANs to generate driving images in adverse conditions of rain and night and apply the generated images to retrain object detection networks. Specifically, we add generated images into the training datasets to retrain the well-known YOLOv5 and evaluate the improvement of the retrained YOLOv5 for object detection in adverse conditions. The experimental results show that the generated driving images by our SUSTechGAN significantly improved the performance of retrained YOLOv5 in rain and night conditions, which outperforms the well-known GANs. The open-source code, video description and datasets are available on the page 1 to facilitate image generation development in autonomous driving under adverse conditions.