CIM-NET: A Video Denoising Deep Neural Network Model Optimized for Computing-in-Memory Architectures

TL;DR

CIM-NET is a novel deep neural network designed specifically for computing-in-memory architectures, enabling efficient real-time video denoising on edge devices by reducing matrix-vector multiplications while maintaining high denoising quality.

Contribution

The paper introduces a hardware-algorithm co-designed framework with CIM-NET architecture and CIM-CONV operator, optimized for CIM chips, significantly reducing inference operations for video denoising.

Findings

Reduces MVM operations to 1/77th of FastDVDnet with minimal performance loss.

Maintains competitive PSNR of 35.11 dB versus 35.56 dB.

Enhances inference speed on CIM hardware for edge video denoising.

Abstract

While deep neural network (DNN)-based video denoising has demonstrated significant performance, deploying state-of-the-art models on edge devices remains challenging due to stringent real-time and energy efficiency requirements. Computing-in-Memory (CIM) chips offer a promising solution by integrating computation within memory cells, enabling rapid matrix-vector multiplication (MVM). However, existing DNN models are often designed without considering CIM architectural constraints, thus limiting their acceleration potential during inference. To address this, we propose a hardware-algorithm co-design framework incorporating two innovations: (1) a CIM-Aware Architecture, CIM-NET, optimized for large receptive field operation and CIM's crossbar-based MVM acceleration; and (2) a pseudo-convolutional operator, CIM-CONV, used within CIM-NET to integrate slide-based processing with fully connected transformations for high-quality feature extraction and reconstruction. This framework significantly reduces the number of MVM operations, improving inference speed on CIM chips while maintaining competitive performance. Experimental results indicate that, compared to the conventional lightweight model FastDVDnet, CIM-NET substantially reduces MVM operations with a slight decrease in denoising performance. With a stride value of 8, CIM-NET reduces MVM operations to 1/77th of the original, while maintaining competitive PSNR (35.11 dB vs. 35.56 dB