Analytic Convolutional Layer: A Step to Analytic Neural Network

TL;DR

The paper introduces the Analytic Convolutional Layer (ACL), a novel convolutional layer combining analytical and traditional kernels, enhancing feature representation, interpretability, and parameter efficiency in neural networks.

Contribution

It proposes the ACL model-driven layer with learnable analytic kernels, improving interpretability and reducing parameters while maintaining high feature representation capacity.

Findings

ACLs achieve better feature representation with fewer parameters.

ACLs increase neural network reliability through analytical formulation.

ACLs facilitate intrinsic interpretability of neural networks.

Abstract

The prevailing approach to embedding prior knowledge within convolutional layers typically includes the design of steerable kernels or their modulation using designated kernel banks. In this study, we introduce the Analytic Convolutional Layer (ACL), an innovative model-driven convolutional layer, which is a mosaic of analytical convolution kernels (ACKs) and traditional convolution kernels. ACKs are characterized by mathematical functions governed by analytic kernel parameters (AKPs) learned in training process. Learnable AKPs permit the adaptive update of incorporated knowledge to align with the features representation of data. Our extensive experiments demonstrate that the ACLs not only have a remarkable capacity for feature representation with a reduced number of parameters but also attain increased reliability through the analytical formulation of ACKs. Furthermore, ACLs offer a means for neural network interpretation, thereby paving the way for the intrinsic interpretability of neural network. The source code will be published in company with the paper.