SegMix: Co-occurrence Driven Mixup for Semantic Segmentation and Adversarial Robustness

TL;DR

SegMix introduces a co-occurrence driven mixup strategy for semantic segmentation that enhances feature binding, improves segmentation accuracy, and increases adversarial robustness by blending images based on category co-occurrence.

Contribution

The paper proposes a novel co-occurrence based image blending method for training segmentation networks, improving both accuracy and robustness against adversarial attacks.

Findings

Enhanced segmentation performance on dense image labeling tasks.

Increased robustness to adversarial attacks.

Effective feature denoising improves prediction accuracy.

Abstract

In this paper, we present a strategy for training convolutional neural networks to effectively resolve interference arising from competing hypotheses relating to inter-categorical information throughout the network. The premise is based on the notion of feature binding, which is defined as the process by which activations spread across space and layers in the network are successfully integrated to arrive at a correct inference decision. In our work, this is accomplished for the task of dense image labelling by blending images based on (i) categorical clustering or (ii) the co-occurrence likelihood of categories. We then train a feature binding network which simultaneously segments and separates the blended images. Subsequent feature denoising to suppress noisy activations reveals additional desirable properties and high degrees of successful predictions. Through this process, we reveal a general mechanism, distinct from any prior methods, for boosting the performance of the base segmentation and saliency network while simultaneously increasing robustness to adversarial attacks.