Semantically Accurate Super-Resolution Generative Adversarial Networks

TL;DR

This paper introduces a novel GAN architecture with a domain-specific feature loss that enhances super-resolution for aerial imagery, significantly improving semantic segmentation accuracy at multiple scales.

Contribution

It presents a joint training approach for super-resolution and semantic segmentation using a new GAN architecture and feature loss, advancing semantic-aware super-resolution techniques.

Findings

Improves perceived image quality and segmentation accuracy.

Achieves 11.8% and 108% accuracy improvements at 4x and 32x super-resolution.

Demonstrates the effectiveness on aerial imagery datasets.

Abstract

This work addresses the problems of semantic segmentation and image super-resolution by jointly considering the performance of both in training a Generative Adversarial Network (GAN). We propose a novel architecture and domain-specific feature loss, allowing super-resolution to operate as a pre-processing step to increase the performance of downstream computer vision tasks, specifically semantic segmentation. We demonstrate this approach using Nearmap's aerial imagery dataset which covers hundreds of urban areas at 5-7 cm per pixel resolution. We show the proposed approach improves perceived image quality as well as quantitative segmentation accuracy across all prediction classes, yielding an average accuracy improvement of 11.8% and 108% at 4x and 32x super-resolution, compared with state-of-the art single-network methods. This work demonstrates that jointly considering image-based and task-specific losses can improve the performance of both, and advances the state-of-the-art in semantic-aware super-resolution of aerial imagery.