Optimal Density Functions for Weighted Convolution in Learning Models

TL;DR

This paper proposes a weighted convolution method using an optimal density function to improve accuracy in image processing tasks, demonstrating significant loss reduction and robustness over standard convolution.

Contribution

It introduces a novel weighted convolution framework with an optimal density function, enhancing CNN performance for image tasks.

Findings

Up to 53% reduction in loss compared to standard convolution

Significant increase in test accuracy

Robustness across various hyperparameters

Abstract

The paper introduces the weighted convolution, a novel approach to the convolution for signals defined on regular grids (e.g., 2D images) through the application of an optimal density function to scale the contribution of neighbouring pixels based on their distance from the central pixel. This choice differs from the traditional uniform convolution, which treats all neighbouring pixels equally. Our weighted convolution can be applied to convolutional neural network problems to improve the approximation accuracy. Given a convolutional network, we define a framework to compute the optimal density function through a minimisation model. The framework separates the optimisation of the convolutional kernel weights (using stochastic gradient descent) from the optimisation of the density function (using DIRECT-L). Experimental results on a learning model for an image-to-image task (e.g., image denoising) show that the weighted convolution significantly reduces the loss (up to 53% improvement) and increases the test accuracy compared to standard convolution. While this method increases execution time by 11%, it is robust across several hyperparameters of the learning model. Future work will apply the weighted convolution to real-case 2D and 3D image convolutional learning problems.