Reverse Convolution and Its Applications to Image Restoration

TL;DR

This paper introduces a novel reverse convolution operator and a Transformer-like block, enabling neural networks to effectively invert convolution operations for improved image restoration tasks.

Contribution

It proposes the first effective depthwise reverse convolution operator and integrates it into a new network architecture called ConverseNet for image restoration.

Findings

Effective reverse convolution operator demonstrated in denoising, super-resolution, and deblurring.

ConverseNet outperforms traditional models in multiple image restoration benchmarks.

The operator can replace standard convolution layers in existing architectures.

Abstract

Convolution and transposed convolution are fundamental operators widely used in neural networks. However, transposed convolution (a.k.a. deconvolution) does not serve as a true inverse of convolution due to inherent differences in their mathematical formulations. To date, no reverse convolution operator has been established as a standard component in neural architectures. In this paper, we propose a novel depthwise reverse convolution operator as an initial attempt to effectively reverse depthwise convolution by formulating and solving a regularized least-squares optimization problem. We thoroughly investigate its kernel initialization, padding strategies, and other critical aspects to ensure its effective implementation. Building upon this operator, we further construct a reverse convolution block by combining it with layer normalization, 1$\times$1 convolution, and GELU activation, forming a Transformer-like structure. The proposed operator and block can directly replace conventional convolution and transposed convolution layers in existing architectures, leading to the development of ConverseNet. Corresponding to typical image restoration models such as DnCNN, SRResNet and USRNet, we train three variants of ConverseNet for Gaussian denoising, super-resolution and deblurring, respectively. Extensive experiments demonstrate the effectiveness of the proposed reverse convolution operator as a basic building module. We hope this work could pave the way for developing new operators in deep model design and applications.