Quadratic Upper Bound for Boosting Robustness

TL;DR

This paper introduces a quadratic upper bound loss function to improve the robustness of fast adversarial training methods, leading to smoother loss landscapes and enhanced defense against adversarial attacks.

Contribution

It proposes a novel quadratic upper bound loss for FAT, significantly improving robustness and smoothing the loss landscape compared to prior methods.

Findings

Enhanced robustness with QUB loss applied to FAT

Smoother loss landscape observed in models using QUB

Significant robustness gains demonstrated across various metrics

Abstract

Fast adversarial training (FAT) aims to enhance the robustness of models against adversarial attacks with reduced training time, however, FAT often suffers from compromised robustness due to insufficient exploration of adversarial space. In this paper, we develop a loss function to mitigate the problem of degraded robustness under FAT. Specifically, we derive a quadratic upper bound (QUB) on the adversarial training (AT) loss function and propose to utilize the bound with existing FAT methods. Our experimental results show that applying QUB loss to the existing methods yields significant improvement of robustness. Furthermore, using various metrics, we demonstrate that this improvement is likely to result from the smoothened loss landscape of the resulting model.