Hierarchical-level rain image generative model based on GAN

TL;DR

This paper introduces RCCycleGAN, a hierarchical rain image generator based on GANs, capable of producing various rain intensities to test autonomous vehicle perception systems, with improved image quality over baseline models.

Contribution

The paper proposes RCCycleGAN, a novel conditional GAN model that generates rain images at different intensities, addressing mode collapse and enhancing image quality for autonomous vehicle testing.

Findings

RCCycleGAN outperforms CycleGAN and DerainCycleGAN in PSNR and SSIM metrics.

The model effectively generates diverse rain intensities for data augmentation.

Ablation studies confirm the effectiveness of model adjustments.

Abstract

Autonomous vehicles are exposed to various weather during operation, which is likely to trigger the performance limitations of the perception system, leading to the safety of the intended functionality (SOTIF) problems. To efficiently generate data for testing the performance of visual perception algorithms under various weather conditions, a hierarchical-level rain image generative model, rain conditional CycleGAN (RCCycleGAN), is constructed. RCCycleGAN is based on the generative adversarial network (GAN) and can generate images of light, medium, and heavy rain. Different rain intensities are introduced as labels in conditional GAN (CGAN). Meanwhile, the model structure is optimized and the training strategy is adjusted to alleviate the problem of mode collapse. In addition, natural rain images of different intensities are collected and processed for model training and validation. Compared with the two baseline models, CycleGAN and DerainCycleGAN, the peak signal-to-noise ratio (PSNR) of RCCycleGAN on the test dataset is improved by 2.58 dB and 0.74 dB, and the structural similarity (SSIM) is improved by 18% and 8%, respectively. The ablation experiments are also carried out to validate the effectiveness of the model tuning.