Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Loss

TL;DR

This paper introduces a novel orthogonal layer and a loss function with annealing to improve the certified robustness of Lipschitz neural networks, resulting in a new model that outperforms existing methods on multiple datasets.

Contribution

The paper proposes a new Block Reflector Orthogonal layer and a logit annealing loss, enhancing Lipschitz neural networks' expressiveness and robustness.

Findings

Achieves state-of-the-art certified robustness on CIFAR-10/100, Tiny-ImageNet, and ImageNet.

Outperforms existing baselines in empirical evaluations.

Provides an efficient and simple architecture for robust deep learning.

Abstract

Lipschitz neural networks are well-known for providing certified robustness in deep learning. In this paper, we present a novel, efficient Block Reflector Orthogonal (BRO) layer that enhances the capability of orthogonal layers on constructing more expressive Lipschitz neural architectures. In addition, by theoretically analyzing the nature of Lipschitz neural networks, we introduce a new loss function that employs an annealing mechanism to increase margin for most data points. This enables Lipschitz models to provide better certified robustness. By employing our BRO layer and loss function, we design BRONet - a simple yet effective Lipschitz neural network that achieves state-of-the-art certified robustness. Extensive experiments and empirical analysis on CIFAR-10/100, Tiny-ImageNet, and ImageNet validate that our method outperforms existing baselines. The implementation is available at https://github.com/ntuaislab/BRONet.