# Depth-aware unpaired image-to-image translation for autonomous driving test scenario generation using a dual-branch GAN

## TL;DR

This paper introduces a new GAN method that uses depth information to create realistic autonomous driving test scenarios, especially in challenging weather conditions.

## Contribution

The novel contribution is the integration of depth information in a dual-branch GAN to improve structural consistency in image translation for autonomous driving.

## Key findings

- DAB-GAN preserves spatial structures during image translation, improving realism in test scenarios.
- The method outperforms existing unpaired translation approaches in visual fidelity and structural integrity.
- It enables robust scenario generation under adverse weather and lighting conditions.

## Abstract

Reliable visual perception is essential for autonomous driving test scenario generation, yet adverse weather and lighting variations pose significant challenges to simulation robustness and generalization. Traditional unpaired image-to-image translation methods primarily rely on RGB-based transformations, often resulting in geometric distortions and loss of structural consistency, which can negatively impact the realism and accuracy of generated test scenarios. To address these limitations, we propose a Depth-Aware Dual-Branch Generative Adversarial Network (DAB-GAN) that explicitly incorporates depth information to preserve spatial structures during scenario generation. The dual-branch generator processes both RGB and depth inputs, ensuring geometric fidelity, while a self-attention mechanism enhances spatial dependencies and local detail refinement. This enables the creation of realistic and structure-preserving test environments that are crucial for evaluating autonomous driving perception systems, especially under adverse weather conditions. Experimental results demonstrate that DAB-GAN outperforms existing unpaired image-to-image translation methods, achieving superior visual fidelity and maintaining depth-aware structural integrity. This approach provides a robust framework for generating diverse and challenging test scenarios, enhancing the development and validation of autonomous driving systems under various real-world conditions.