On an Interpretation of ResNets via Solution Constructions

TL;DR

This paper provides a theoretical interpretation of ResNet architectures through solution constructions, explaining their performance mechanism and proving their universal approximation capability.

Contribution

It introduces a novel solution-based interpretation of ResNets and demonstrates their universal approximation property.

Findings

ResNet solutions can be constructed via gate-network controls.

The interpretation explains ResNet's performance mechanism.

ResNets are proven to have universal approximation capabilities.

Abstract

This paper first constructs a typical solution of ResNets for multi-category classifications by the principle of gate-network controls and deep-layer classifications, from which a general interpretation of the ResNet architecture is given and the performance mechanism is explained. We then use more solutions to further demonstrate the generality of that interpretation. The universal-approximation capability of ResNets is proved.