Explicitising The Implicit Intrepretability of Deep Neural Networks Via Duality

TL;DR

This paper explores the implicit interpretability of deep neural networks through a duality perspective, revealing new theoretical properties and proposing a novel network class that enhances interpretability without sacrificing performance.

Contribution

It introduces the concept of deep linearly gated networks (DLGNs) and demonstrates their improved interpretability-accuracy tradeoff using the neural path kernel and duality theory.

Findings

Weights can be trained with a constant input

Gating masks can be shuffled without performance loss

DLGNs outperform traditional networks in interpretability-accuracy tradeoff

Abstract

Recent work by Lakshminarayanan and Singh [2020] provided a dual view for fully connected deep neural networks (DNNs) with rectified linear units (ReLU). It was shown that (i) the information in the gates is analytically characterised by a kernel called the neural path kernel (NPK) and (ii) most critical information is learnt in the gates, in that, given the learnt gates, the weights can be retrained from scratch without significant loss in performance. Using the dual view, in this paper, we rethink the conventional interpretations of DNNs thereby explicitsing the implicit interpretability of DNNs. Towards this, we first show new theoretical properties namely rotational invariance and ensemble structure of the NPK in the presence of convolutional layers and skip connections respectively. Our theory leads to two surprising empirical results that challenge conventional wisdom: (i) the weights can be trained even with a constant 1 input, (ii) the gating masks can be shuffled, without any significant loss in performance. These results motivate a novel class of networks which we call deep linearly gated networks (DLGNs). DLGNs using the phenomenon of dual lifting pave way to more direct and simpler interpretation of DNNs as opposed to conventional interpretations. We show via extensive experiments on CIFAR-10 and CIFAR-100 that these DLGNs lead to much better interpretability-accuracy tradeoff.