A Hardware-Friendly Joint Denoising and Demosaicing System Based on Efficient FPGA Implementation

TL;DR

This paper presents an efficient FPGA-based system for joint image denoising and demosaicing with reduced computational complexity and improved performance.

Contribution

A lightweight neural network with partial convolution and a flexible FPGA hardware platform for efficient joint image processing.

Findings

The proposed system reduces model parameters and MACs by 83.38% and 77.71%, respectively.

It achieves a 2.36dB PSNR and 0.0806 SSIM improvement over state-of-the-art methods.

The hardware supports multi-parallelism and adapts to edge-embedded scenarios.

Abstract

This paper designs a hardware-implementable joint denoising and demosaicing acceleration system. Firstly, a lightweight network architecture with multi-scale feature extraction based on partial convolution is proposed at the algorithm level. The partial convolution scheme can reduce the redundancy of filters and feature maps, thereby reducing memory accesses, and achieve excellent visual effects with a smaller model complexity. In addition, multi-scale extraction can expand the receptive field while reducing model parameters. Then, we apply separable convolution and partial convolution to reduce the parameters of the model. Compared with the standard convolutional solution, the parameters and MACs are reduced by 83.38% and 77.71%, respectively. Moreover, different networks bring different memory access and complex computing methods; thus, we introduce a unified and flexibly configurable hardware acceleration processing platform and implement it on the Xilinx Zynq UltraScale + FPGA board. Finally, compared with the state-of-the-art neural network solution on the Kodak24 set, the peak signal-to-noise ratio and the structural similarity index measure are approximately improved by 2.36dB and 0.0806, respectively, and the computing efficiency is improved by 2.09×. Furthermore, the hardware architecture supports multi-parallelism and can adapt to the different edge-embedded scenarios. Overall, the image processing task solution proposed in this paper has positive advantages in the joint denoising and demosaicing system.

Figures