Reimagine BiSeNet for Real-Time Domain Adaptation in Semantic Segmentation

TL;DR

This paper introduces a lightweight adversarial method for real-time domain adaptation in semantic segmentation, enabling models trained on synthetic data to perform effectively on real-world data with minimal resources.

Contribution

It presents the first real-time adversarial approach for domain adaptation in semantic segmentation using a novel lightweight discriminator.

Findings

Effective adaptation from synthetic to real data in real-time

Achieved competitive performance on GTA5 to Cityscapes and SYNTHIA to Cityscapes benchmarks

Demonstrated suitability for embedded and resource-constrained devices

Abstract

Semantic segmentation models have reached remarkable performance across various tasks. However, this performance is achieved with extremely large models, using powerful computational resources and without considering training and inference time. Real-world applications, on the other hand, necessitate models with minimal memory demands, efficient inference speed, and executable with low-resources embedded devices, such as self-driving vehicles. In this paper, we look at the challenge of real-time semantic segmentation across domains, and we train a model to act appropriately on real-world data even though it was trained on a synthetic realm. We employ a new lightweight and shallow discriminator that was specifically created for this purpose. To the best of our knowledge, we are the first to present a real-time adversarial approach for assessing the domain adaption problem in semantic segmentation. We tested our framework in the two standard protocol: GTA5 to Cityscapes and SYNTHIA to Cityscapes. Code is available at: https://github.com/taveraantonio/RTDA.