SyntStereo2Real: Edge-Aware GAN for Remote Sensing Image-to-Image Translation while Maintaining Stereo Constraint

TL;DR

This paper introduces SyntStereo2Real, an edge-aware GAN that improves remote sensing image translation by maintaining stereo constraints, enhancing domain generalization and geometric consistency.

Contribution

It presents a novel edge-aware GAN architecture that simultaneously performs image translation and stereo-matching with stereo constraint enforcement.

Findings

Produces superior qualitative results

Achieves better quantitative performance

Effective across diverse domains like autonomous driving

Abstract

In the field of remote sensing, the scarcity of stereo-matched and particularly lack of accurate ground truth data often hinders the training of deep neural networks. The use of synthetically generated images as an alternative, alleviates this problem but suffers from the problem of domain generalization. Unifying the capabilities of image-to-image translation and stereo-matching presents an effective solution to address the issue of domain generalization. Current methods involve combining two networks, an unpaired image-to-image translation network and a stereo-matching network, while jointly optimizing them. We propose an edge-aware GAN-based network that effectively tackles both tasks simultaneously. We obtain edge maps of input images from the Sobel operator and use it as an additional input to the encoder in the generator to enforce geometric consistency during translation. We additionally include a warping loss calculated from the translated images to maintain the stereo consistency. We demonstrate that our model produces qualitatively and quantitatively superior results than existing models, and its applicability extends to diverse domains, including autonomous driving.