Operational vs Convolutional Neural Networks for Image Denoising

TL;DR

This paper introduces Operational Neural Networks (ONNs), a flexible and heterogeneous neural network model that outperforms traditional CNNs in image denoising tasks by incorporating diverse non-linear operators.

Contribution

The paper proposes ONNs with heterogeneous non-linearities and a data-driven operator search strategy, advancing beyond the limitations of conventional CNNs in image denoising.

Findings

ONNs outperform CNNs in image denoising tasks

Heterogeneous non-linear operators improve denoising quality

Operator search strategy effectively selects optimal non-linearities

Abstract

Convolutional Neural Networks (CNNs) have recently become a favored technique for image denoising due to its adaptive learning ability, especially with a deep configuration. However, their efficacy is inherently limited owing to their homogenous network formation with the unique use of linear convolution. In this study, we propose a heterogeneous network model which allows greater flexibility for embedding additional non-linearity at the core of the data transformation. To this end, we propose the idea of an operational neuron or Operational Neural Networks (ONN), which enables a flexible non-linear and heterogeneous configuration employing both inter and intra-layer neuronal diversity. Furthermore, we propose a robust operator search strategy inspired by the Hebbian theory, called the Synaptic Plasticity Monitoring (SPM) which can make data-driven choices for non-linearities in any architecture. An extensive set of comparative evaluations of ONNs and CNNs over two severe image denoising problems yield conclusive evidence that ONNs enriched by non-linear operators can achieve a superior denoising performance against CNNs with both equivalent and well-known deep configurations.