Lookahead Adversarial Learning for Near Real-Time Semantic Segmentation

TL;DR

This paper introduces a lookahead adversarial learning approach with label map aggregation to improve the stability and performance of real-time semantic segmentation models in adversarial settings.

Contribution

It presents a novel lookahead adversarial training method with label aggregation, enabling stable adversarial learning with fast semantic segmentation models.

Findings

Achieved +5% performance improvement on standard datasets.

Alleviated divergence issues in adversarial semantic segmentation.

Enhanced stability of near real-time segmentation models in adversarial training.

Abstract

Semantic segmentation is one of the most fundamental problems in computer vision with significant impact on a wide variety of applications. Adversarial learning is shown to be an effective approach for improving semantic segmentation quality by enforcing higher-level pixel correlations and structural information. However, state-of-the-art semantic segmentation models cannot be easily plugged into an adversarial setting because they are not designed to accommodate convergence and stability issues in adversarial networks. We bridge this gap by building a conditional adversarial network with a state-of-the-art segmentation model (DeepLabv3+) at its core. To battle the stability issues, we introduce a novel lookahead adversarial learning (LoAd) approach with an embedded label map aggregation module. We focus on semantic segmentation models that run fast at inference for near real-time field applications. Through extensive experimentation, we demonstrate that the proposed solution can alleviate divergence issues in an adversarial semantic segmentation setting and results in considerable performance improvements (+5% in some classes) on the baseline for three standard datasets.