SyReNN: A Tool for Analyzing Deep Neural Networks

TL;DR

SyReNN is a novel tool that analyzes deep neural networks by computing their symbolic representations through decomposition into linear functions, enabling targeted analysis of input subsets.

Contribution

The paper introduces SyReNN, a new tool for analyzing DNNs via symbolic representation and linear decomposition, focusing on low-dimensional input subsets.

Findings

Effective in computing Integrated Gradients

Visualizes DNN decision boundaries

Assists in patching neural networks

Abstract

Deep Neural Networks (DNNs) are rapidly gaining popularity in a variety of important domains. Formally, DNNs are complicated vector-valued functions which come in a variety of sizes and applications. Unfortunately, modern DNNs have been shown to be vulnerable to a variety of attacks and buggy behavior. This has motivated recent work in formally analyzing the properties of such DNNs. This paper introduces SyReNN, a tool for understanding and analyzing a DNN by computing its symbolic representation. The key insight is to decompose the DNN into linear functions. Our tool is designed for analyses using low-dimensional subsets of the input space, a unique design point in the space of DNN analysis tools. We describe the tool and the underlying theory, then evaluate its use and performance on three case studies: computing Integrated Gradients, visualizing a DNN's decision boundaries, and patching a DNN.