An Interpretive Constrained Linear Model for ResNet and MgNet

TL;DR

This paper introduces an interpretable constrained linear model for CNNs like ResNet and MgNet, establishing theoretical connections, proposing modifications, and demonstrating improved accuracy and efficiency in image classification.

Contribution

It presents a new constrained linear data-feature-mapping model, links it to existing architectures, and proposes modifications that improve performance with fewer parameters.

Findings

Modified ResNet models with fewer parameters outperform original models.

MgNet demonstrates superior accuracy and efficiency in image classification.

Systematic numerical studies validate MgNet's advantages over established networks.

Abstract

We propose a constrained linear data-feature-mapping model as an interpretable mathematical model for image classification using a convolutional neural network (CNN). From this viewpoint, we establish detailed connections between the traditional iterative schemes for linear systems and the architectures of the basic blocks of ResNet- and MgNet-type models. Using these connections, we present some modified ResNet models that compared with the original models have fewer parameters and yet can produce more accurate results, thereby demonstrating the validity of this constrained learning data-feature-mapping assumption. Based on this assumption, we further propose a general data-feature iterative scheme to show the rationality of MgNet. We also provide a systematic numerical study on MgNet to show its success and advantages in image classification problems and demonstrate its advantages in comparison with established networks.