Multigrid-in-Channels Architectures for Wide Convolutional Neural Networks

TL;DR

This paper introduces a multigrid-in-channels approach for CNNs that reduces parameter growth and maintains accuracy when widening networks, by replacing standard convolutions with structured multilevel layers.

Contribution

It proposes a novel multigrid-in-channels architecture that achieves linear parameter scaling and full channel coupling, improving efficiency over traditional CNNs.

Findings

Reduces parameters in residual networks and MobileNetV2.

Maintains accuracy despite significant parameter reduction.

Enables wider networks without increased computational cost.

Abstract

We present a multigrid approach that combats the quadratic growth of the number of parameters with respect to the number of channels in standard convolutional neural networks (CNNs). It has been shown that there is a redundancy in standard CNNs, as networks with much sparser convolution operators can yield similar performance to full networks. The sparsity patterns that lead to such behavior, however, are typically random, hampering hardware efficiency. In this work, we present a multigrid-in-channels approach for building CNN architectures that achieves full coupling of the channels, and whose number of parameters is linearly proportional to the width of the network. To this end, we replace each convolution layer in a generic CNN with a multilevel layer consisting of structured (i.e., grouped) convolutions. Our examples from supervised image classification show that applying this strategy to residual networks and MobileNetV2 considerably reduces the number of parameters without negatively affecting accuracy. Therefore, we can widen networks without dramatically increasing the number of parameters or operations.