Robustness Evaluation and Adversarial Training of an Instance Segmentation Model

TL;DR

This paper introduces probabilistic local equivalence for evaluating robustness of non-classifier models and demonstrates that adversarial training improves robustness in instance segmentation without sacrificing training performance.

Contribution

It proposes a new robustness evaluation method for non-classifier models and applies adversarial training to enhance robustness in instance segmentation tasks.

Findings

Achieved a dice score of 0.85 on TuSimple challenge.

Obtained an F-measure of 0.49 on manipulated inputs.

Probabilistic local equivalence distinguishes trained models' robustness.

Abstract

To evaluate the robustness of non-classifier models, we propose probabilistic local equivalence, based on the notion of randomized smoothing, as a way to quantitatively evaluate the robustness of an arbitrary function. In addition, to understand the effect of adversarial training on non-classifiers and to investigate the level of robustness that can be obtained without degrading performance on the training distribution, we apply Fast is Better than Free adversarial training together with the TRADES robust loss to the training of an instance segmentation network. In this direction, we were able to achieve a symmetric best dice score of 0.85 on the TuSimple lane detection challenge, outperforming the standardly-trained network's score of 0.82. Additionally, we were able to obtain an F-measure of 0.49 on manipulated inputs, in contrast to the standardly-trained network's score of 0. We show that probabilisitic local equivalence is able to successfully distinguish between standardly-trained and adversarially-trained models, providing another view of the improved robustness of the adversarially-trained models.