Efficient Meta-Tuning for Content-aware Neural Video Delivery

TL;DR

This paper introduces Efficient Meta-Tuning (EMT), a novel approach that significantly reduces the computational cost of content-aware neural video delivery by meta-learning and selective fine-tuning, enabling practical application of neural video streaming.

Contribution

The paper proposes EMT, a meta-learning based method that adapts models efficiently for each video chunk, reducing training time and computational cost while improving performance.

Findings

EMT reduces training time compared to traditional methods.

The method outperforms existing approaches in quality and efficiency.

It generalizes well across various super-resolution architectures.

Abstract

Recently, Deep Neural Networks (DNNs) are utilized to reduce the bandwidth and improve the quality of Internet video delivery. Existing methods train corresponding content-aware super-resolution (SR) model for each video chunk on the server, and stream low-resolution (LR) video chunks along with SR models to the client. Although they achieve promising results, the huge computational cost of network training limits their practical applications. In this paper, we present a method named Efficient Meta-Tuning (EMT) to reduce the computational cost. Instead of training from scratch, EMT adapts a meta-learned model to the first chunk of the input video. As for the following chunks, it fine-tunes the partial parameters selected by gradient masking of previous adapted model. In order to achieve further speedup for EMT, we propose a novel sampling strategy to extract the most challenging patches from video frames. The proposed strategy is highly efficient and brings negligible additional cost. Our method significantly reduces the computational cost and achieves even better performance, paving the way for applying neural video delivery techniques to practical applications. We conduct extensive experiments based on various efficient SR architectures, including ESPCN, SRCNN, FSRCNN and EDSR-1, demonstrating the generalization ability of our work. The code is released at \url{https://github.com/Neural-video-delivery/EMT-Pytorch-ECCV2022}.