IC-Network: Efficient Structure for Convolutional Neural Networks

TL;DR

This paper introduces the IC-Network, a novel structure inspired by physics that enhances CNNs by integrating the Inter-layer Collision (IC) structure, leading to faster training and better generalization.

Contribution

The paper proposes the IC structure and computational units, which improve CNN performance and training efficiency by adding non-linear representation capabilities.

Findings

IC structure improves CNN accuracy on ImageNet

IC block reduces top-1 error in ResNet-50 from 22.85% to 21.49%

IC units outperform traditional convolutions in feature filtering

Abstract

Neural networks have been widely used, and most networks achieve excellent performance by stacking certain types of basic units. Compared to increasing the depth and width of the network, designing more effective basic units has become an important research topic. Inspired by the elastic collision model in physics, we present a universal structure that could be integrated into the existing network structures to speed up the training process and increase their generalization abilities. We term this structure the "Inter-layer Collision" (IC) structure. We built two kinds of basic computational units (IC layer and IC block) that compose the convolutional neural networks (CNNs) by combining the IC structure with the convolution operation. Compared to traditional convolutions, both of the proposed computational units have a stronger non-linear representation ability and can filter features useful for a given task. Using these computational units to build networks, we bring significant improvements in performance for existing state-of-the-art CNNs. On the imagenet experiment, we integrate the IC block into ResNet-50 and reduce the top-1 error from 22.85% to 21.49%, which also exceeds the top-1 error of ResNet-100 (21.75%).