HOPE: High-order Polynomial Expansion of Black-box Neural Networks

TL;DR

HOPE introduces a high-order Taylor polynomial expansion method for neural networks, enabling explicit local interpretability, faster inference, and improved feature analysis with high accuracy and low computational cost.

Contribution

The paper develops a novel high-order derivative rule for neural networks, allowing efficient Taylor polynomial expansion for interpretability and applications in function discovery and feature selection.

Findings

High accuracy in neural network approximation

Low computational complexity of the method

Effective applications in interpretability and feature analysis

Abstract

Despite their remarkable performance, deep neural networks remain mostly ``black boxes'', suggesting inexplicability and hindering their wide applications in fields requiring making rational decisions. Here we introduce HOPE (High-order Polynomial Expansion), a method for expanding a network into a high-order Taylor polynomial on a reference input. Specifically, we derive the high-order derivative rule for composite functions and extend the rule to neural networks to obtain their high-order derivatives quickly and accurately. From these derivatives, we can then derive the Taylor polynomial of the neural network, which provides an explicit expression of the network's local interpretations. Numerical analysis confirms the high accuracy, low computational complexity, and good convergence of the proposed method. Moreover, we demonstrate HOPE's wide applications built on deep learning, including function discovery, fast inference, and feature selection. The code is available at https://github.com/HarryPotterXTX/HOPE.git.