ERNet Family: Hardware-Oriented CNN Models for Computational Imaging Using Block-Based Inference

TL;DR

This paper introduces ERNet, a family of hardware-aware CNN models optimized for block-based computational imaging, reducing bandwidth and hardware costs while maintaining high image quality.

Contribution

The paper proposes ERNet, a novel CNN model family with temporary layer expansion, optimized for hardware constraints in block-based imaging tasks.

Findings

ERNet outperforms FFDNet and EDSR-baseline in denoising and super-resolution.

ERNet achieves high image quality with reduced SRAM usage.

The hardware-aware optimization improves efficiency in Full HD and 4K applications.

Abstract

Convolutional neural networks (CNNs) demand huge DRAM bandwidth for computational imaging tasks, and block-based processing has recently been applied to greatly reduce the bandwidth. However, the induced additional computation for feature recomputing or the large SRAM for feature reusing will degrade the performance or even forbid the usage of state-of-the-art models. In this paper, we address these issues by considering the overheads and hardware constraints in advance when constructing CNNs. We investigate a novel model family---ERNet---which includes temporary layer expansion as another means for increasing model capacity. We analyze three ERNet variants in terms of hardware requirement and introduce a hardware-aware model optimization procedure. Evaluations on Full HD and 4K UHD applications will be given to show the effectiveness in terms of image quality, pixel throughput, and SRAM usage. The results also show that, for block-based inference, ERNet can outperform the state-of-the-art FFDNet and EDSR-baseline models for image denoising and super-resolution respectively.