Overparametrization of HyperNetworks at Fixed FLOP-Count Enables Fast Neural Image Enhancement

TL;DR

This paper introduces a method using HyperNetworks to surpass existing neural image enhancement models in quality while significantly reducing computational cost, demonstrating improved generalization and efficiency on multiple datasets.

Contribution

The paper proposes a novel use of HyperNetworks to decouple FLOPs from parameters in convolutional networks, enabling faster image enhancement with better quality at lower computational costs.

Findings

Achieved state-of-the-art SSIM and MS-SSIM on ZRR dataset with over 10x fewer FLOPs.

Observed double-descent behavior in generalization curves at fixed FLOP-count.

Reduced computational cost of existing networks like VDN while maintaining image fidelity.

Abstract

Deep convolutional neural networks can enhance images taken with small mobile camera sensors and excel at tasks like demoisaicing, denoising and super-resolution. However, for practical use on mobile devices these networks often require too many FLOPs and reducing the FLOPs of a convolution layer, also reduces its parameter count. This is problematic in view of the recent finding that heavily over-parameterized neural networks are often the ones that generalize best. In this paper we propose to use HyperNetworks to break the fixed ratio of FLOPs to parameters of standard convolutions. This allows us to exceed previous state-of-the-art architectures in SSIM and MS-SSIM on the Zurich RAW- to-DSLR (ZRR) data-set at > 10x reduced FLOP-count. On ZRR we further observe generalization curves consistent with 'double-descent' behavior at fixed FLOP-count, in the large image limit. Finally we demonstrate the same technique can be applied to an existing network (VDN) to reduce its computational cost while maintaining fidelity on the Smartphone Image Denoising Dataset (SIDD). Code for key functions is given in the appendix.