Adversarially robust segmentation models learn perceptually-aligned gradients

TL;DR

This paper demonstrates that adversarially-trained semantic segmentation models produce perceptually-aligned gradients, leading to more robust and interpretable image inpainting and generation.

Contribution

It shows that adversarial training enhances segmentation models' robustness and gradient interpretability, enabling effective image synthesis tasks.

Findings

Adversarially-trained segmentation models are more robust.

They exhibit perceptually-aligned gradients.

Enhanced interpretability in image inpainting.

Abstract

The effects of adversarial training on semantic segmentation networks has not been thoroughly explored. While previous work has shown that adversarially-trained image classifiers can be used to perform image synthesis, we have yet to understand how best to leverage an adversarially-trained segmentation network to do the same. Using a simple optimizer, we demonstrate that adversarially-trained semantic segmentation networks can be used to perform image inpainting and generation. Our experiments demonstrate that adversarially-trained segmentation networks are more robust and indeed exhibit perceptually-aligned gradients which help in producing plausible image inpaintings. We seek to place additional weight behind the hypothesis that adversarially robust models exhibit gradients that are more perceptually-aligned with human vision. Through image synthesis, we argue that perceptually-aligned gradients promote a better understanding of a neural network's learned representations and aid in making neural networks more interpretable.