Interactive Analysis of CNN Robustness

TL;DR

Perturber is a web tool enabling interactive exploration of CNN robustness against various scene perturbations, aiding in identifying vulnerabilities and comparing model behaviors.

Contribution

Introduces Perturber, a web-based application for real-time analysis of CNN predictions under diverse scene modifications and adversarial attacks.

Findings

Helps users generate hypotheses about CNN vulnerabilities.

Enables qualitative comparison of different CNN models.

Replicates user insights with quantitative analysis.

Abstract

While convolutional neural networks (CNNs) have found wide adoption as state-of-the-art models for image-related tasks, their predictions are often highly sensitive to small input perturbations, which the human vision is robust against. This paper presents Perturber, a web-based application that allows users to instantaneously explore how CNN activations and predictions evolve when a 3D input scene is interactively perturbed. Perturber offers a large variety of scene modifications, such as camera controls, lighting and shading effects, background modifications, object morphing, as well as adversarial attacks, to facilitate the discovery of potential vulnerabilities. Fine-tuned model versions can be directly compared for qualitative evaluation of their robustness. Case studies with machine learning experts have shown that Perturber helps users to quickly generate hypotheses about model vulnerabilities and to qualitatively compare model behavior. Using quantitative analyses, we could replicate users' insights with other CNN architectures and input images, yielding new insights about the vulnerability of adversarially trained models.