EVRNet: Efficient Video Restoration on Edge Devices

TL;DR

EVRNet is a lightweight, efficient neural network designed for real-time video restoration on edge devices, achieving competitive quality with significantly fewer parameters and computations across various restoration tasks.

Contribution

The paper introduces EVRNet, a novel video restoration network that uses parameter-efficient modules, enabling real-time processing on edge devices with minimal resource usage.

Findings

EVRNet achieves comparable SSIM scores to larger models with far fewer parameters.

It performs effectively on multiple restoration tasks like deblocking, denoising, and super-resolution.

EVRNet demonstrates robustness across different distortions and motion scenarios.

Abstract

Video transmission applications (e.g., conferencing) are gaining momentum, especially in times of global health pandemic. Video signals are transmitted over lossy channels, resulting in low-quality received signals. To restore videos on recipient edge devices in real-time, we introduce an efficient video restoration network, EVRNet. EVRNet efficiently allocates parameters inside the network using alignment, differential, and fusion modules. With extensive experiments on video restoration tasks (deblocking, denoising, and super-resolution), we demonstrate that EVRNet delivers competitive performance to existing methods with significantly fewer parameters and MACs. For example, EVRNet has 260 times fewer parameters and 958 times fewer MACs than enhanced deformable convolution-based video restoration network (EDVR) for 4 times video super-resolution while its SSIM score is 0.018 less than EDVR. We also evaluated the performance of EVRNet under multiple distortions on unseen dataset to demonstrate its ability in modeling variable-length sequences under both camera and object motion.