Improving Convolutional Neural Networks Via Conservative Field Regularisation and Integration

TL;DR

This paper introduces a novel CNN operation based on Green's functions that regularizes feature fields to be conservative, leading to faster learning and better generalization without extra parameters.

Contribution

It proposes a new operation using Green's functions for CNNs to enforce conservative fields, enhancing learning and generalization.

Findings

Faster training convergence

Smoother training curves

Improved generalization

Abstract

Current research in convolutional neural networks (CNN) focuses mainly on changing the architecture of the networks, optimizing the hyper-parameters and improving the gradient descent. However, most work use only 3 standard families of operations inside the CNN, the convolution, the activation function, and the pooling. In this work, we propose a new family of operations based on the Green's function of the Laplacian, which allows the network to solve the Laplacian, to integrate any vector field and to regularize the field by forcing it to be conservative. Hence, the Green's function (GF) is the first operation that regularizes the 2D or 3D feature space by forcing it to be conservative and physically interpretable, instead of regularizing the norm of the weights. Our results show that such regularization allows the network to learn faster, to have smoother training curves and to better generalize, without any additional parameter. The current manuscript presents early results, more work is required to benchmark the proposed method.