Library network, a possible path to explainable neural networks

TL;DR

This paper proposes a new algorithm that enhances understanding of deep neural networks' decision processes and detects adversarial attacks, addressing transparency and vulnerability issues in high-stakes applications.

Contribution

The paper introduces an algorithm that traces DNN decision pathways across layers and identifies adversarial attacks, improving interpretability and robustness.

Findings

Algorithm effectively traces decision processes across layers

Detects adversarial attacks reliably

Improves transparency of DNN decision-making

Abstract

Deep neural networks (DNNs) may outperform human brains in complex tasks, but the lack of transparency in their decision-making processes makes us question whether we could fully trust DNNs with high stakes problems. As DNNs' operations rely on a massive number of both parallel and sequential linear/nonlinear computations, predicting their mistakes is nearly impossible. Also, a line of studies suggests that DNNs can be easily deceived by adversarial attacks, indicating that their decisions can easily be corrupted by unexpected factors. Such vulnerability must be overcome if we intend to take advantage of DNNs' efficiency in high stakes problems. Here, we propose an algorithm that can help us better understand DNNs' decision-making processes. Our empirical evaluations suggest that this algorithm can effectively trace DNNs' decision processes from one layer to another and detect adversarial attacks.