Interpreting Robustness Proofs of Deep Neural Networks

TL;DR

This paper develops methods to generate human-understandable interpretations of robustness proofs for deep neural networks, revealing reliance on spurious features and improvements with combined training methods.

Contribution

It introduces new concepts and algorithms for interpretable robustness proofs, bridging the gap between formal verification and human understanding.

Findings

Standard DNNs rely on spurious features for robustness proofs.

Provably robust DNNs filter out meaningful features in proofs.

Combined adversarial and provable training enhances interpretability and feature filtering.

Abstract

In recent years numerous methods have been developed to formally verify the robustness of deep neural networks (DNNs). Though the proposed techniques are effective in providing mathematical guarantees about the DNNs behavior, it is not clear whether the proofs generated by these methods are human-interpretable. In this paper, we bridge this gap by developing new concepts, algorithms, and representations to generate human understandable interpretations of the proofs. Leveraging the proposed method, we show that the robustness proofs of standard DNNs rely on spurious input features, while the proofs of DNNs trained to be provably robust filter out even the semantically meaningful features. The proofs for the DNNs combining adversarial and provably robust training are the most effective at selectively filtering out spurious features as well as relying on human-understandable input features.