Towards the Dynamics of a DNN Learning Symbolic Interactions

TL;DR

This paper mathematically proves the two-phase dynamics of interactions in deep neural networks, explaining how their generalization power evolves during training and providing a theoretical foundation for understanding DNN inference logic.

Contribution

It introduces a formal proof of the two-phase interaction dynamics in DNNs, linking these phases to changes in generalization during training.

Findings

Theory accurately predicts interaction dynamics across different DNNs

Two-phase dynamics explain the transition from under-fitting to over-fitting

Provides a mathematical framework for DNN inference mechanisms

Abstract

This study proves the two-phase dynamics of a deep neural network (DNN) learning interactions. Despite the long disappointing view of the faithfulness of post-hoc explanation of a DNN, a series of theorems have been proven in recent years to show that for a given input sample, a small set of interactions between input variables can be considered as primitive inference patterns that faithfully represent a DNN's detailed inference logic on that sample. Particularly, Zhang et al. have observed that various DNNs all learn interactions of different complexities in two distinct phases, and this two-phase dynamics well explains how a DNN changes from under-fitting to over-fitting. Therefore, in this study, we mathematically prove the two-phase dynamics of interactions, providing a theoretical mechanism for how the generalization power of a DNN changes during the training process. Experiments show that our theory well predicts the real dynamics of interactions on different DNNs trained for various tasks.