Achieving Domain-Independent Certified Robustness via Knowledge Continuity

TL;DR

This paper introduces knowledge continuity, a new concept inspired by Lipschitz continuity, to certify neural network robustness across diverse input domains with guarantees based solely on the loss function and learned metrics.

Contribution

It proposes a domain-independent robustness certification method based on knowledge continuity, extending robustness guarantees beyond traditional norm-based approaches.

Findings

Certification guarantees are independent of domain modality and distribution.

Maximizing knowledge continuity does not compromise model expressiveness.

Applications include regularization, certification algorithms, and vulnerability localization.

Abstract

We present knowledge continuity, a novel definition inspired by Lipschitz continuity which aims to certify the robustness of neural networks across input domains (such as continuous and discrete domains in vision and language, respectively). Most existing approaches that seek to certify robustness, especially Lipschitz continuity, lie within the continuous domain with norm and distribution-dependent guarantees. In contrast, our proposed definition yields certification guarantees that depend only on the loss function and the intermediate learned metric spaces of the neural network. These bounds are independent of domain modality, norms, and distribution. We further demonstrate that the expressiveness of a model class is not at odds with its knowledge continuity. This implies that achieving robustness by maximizing knowledge continuity should not theoretically hinder inferential performance. Finally, to complement our theoretical results, we present several applications of knowledge continuity such as regularization, a certification algorithm, and show that knowledge continuity can be used to localize vulnerable components of a neural network.